Neuronal convergence in early contrast vision: Binocular summation is followed by response nonlinearity and area summation

Abstract

We assessed summation of contrast across eyes and area at detection threshold ( C _t). Stimuli were sine-wave gratings (2.5 c/deg) spatially modulated by cosine- and anticosine-phase raised plaids (0.5 c/deg components oriented at ±45°). When presented dichoptically the signal regions were interdigitated across eyes but produced a smooth continuous grating following their linear binocular sum. The average summation ratio ( C _t1/([ C _t1+2]) for this stimulus pair was 1.64 (4.3 dB). This was only slightly less than the binocular summation found for the same patch type presented to both eyes, and the area summation found for the two different patch types presented to the same eye. We considered 192 model architectures containing each of the following four elements in all possible orders: (i) linear summation or a MAX operator across eyes, (ii) linear summation or a MAX operator across area, (iii) linear or accelerating contrast transduction, and (iv) additive Gaussian, stochastic noise. Formal equivalences reduced this to 62 different models. The most successful four-element model was: linear summation across eyes followed by nonlinear contrast transduction, linear summation across area, and late noise. Model performance was enhanced when additional nonlinearities were placed before binocular summation and after area summation. The implications for models of probability summation and uncertainty are discussed.

Introduction

The initial stages of vision decompose the two retinal images into local estimates of feature dimensions such as contrast, size, and orientation. However, (i) normal observers experience a unitary (binocular) vision of the world and (ii) the world contains spatially extensive surfaces and textures whose projections exceed the footprints (receptive fields) of the local retinal analyses, at least up to layer 4 of V1. Neuronal convergence across space and eyes is a necessary condition for building binocular object representations from local monocular measures, but what is the form of the convergence, and how is it organized? 

One way in which this can be investigated is to measure contrast detection thresholds as a function of the dimension of interest, and assess the level of improvement against various models of the process (Foley, Varadharajan, Koh, & Farias, 2007; Kersten, 1984; Meese, Georgeson, & Baker, 2006; Robson & Graham, 1981; Watson, 1979). However, a difficulty is that the number of model parameters or potential architectures is not necessarily well constrained by this approach. For example, in experiments that increase the size of a patch of grating placed in the central visual field, potentially confounding variables include: the level of noise, retinal inhomogeneity, and uncertainty. Untangling these parameters poses a serious challenge to interpretation of this kind of experiment. 

In a recent study, Meese and Summers (2007) introduced a stimulus set that was designed to overcome these problems in the spatial (area) domain. The basic idea was to use a grating-type stimulus with a constant diameter to encourage contrast integration (by whatever means) over the same retinal mechanisms in all conditions. If this could be achieved, then it seemed likely that this would control all of the problems outlined above. But how can the diameter of the stimulus be fixed, while allowing its area to be varied? The answer was to cut holes in the stimulus, or more accurately, to attenuate interdigitated patches of the stimulus. Example stimuli are shown in Figure 1. Figure 1a is a sine-wave grating that has been modulated by a ‘raised plaid’ (see Methods section for details) in cosine phase with the center of the stimulus. Figure 1b is similar, but the modulation is in anticosine phase. These stimuli were given the nominal titles of ‘white’ and ‘black’ checks, respectively, as a reference to the magnitude of the modulator at the center of the display (unity and zero). Figure 1c is the physical sum of the stimuli in Figures 1a and 1b and is referred to as the ‘full’ stimulus. It is also the original sine-wave grating without modulation by the raised plaid. Note then, the sum of contrast over area (we refer to this as ‘contrast area’) is the same in Figures 1a and 1b, which is half that in Figure 1c. 

Figure 1

 

Stimuli introduced by Meese and Summers (2007) and used in the three experiments here. In (a) and (b) the sine-wave grating in (c) has been multiplied by a ‘raised plaid’ (see Methods section) in cosine and anticosine phase with the center of the display, respectively. The nominal titles in (a) and (b) refer to the modulator in the center of the display. All three stimulus types were used in the ‘area’ experiment (summation across space), but only those in (a) and (b) were used in the ‘eyes’ experiment (binocular summation) and the ‘eyes and area’ experiment (summation across eyes and area together).

At contrast detection threshold, Meese and Summers (2007) found that sensitivities to the ‘black’ and ‘white’ checks were identical, but that sensitivity to the full stimulus was almost twice as great (a factor of 1.8). Their modeling led them to conclude that vision can perform linear summation (after nonlinear contrast transduction) over at least seven cycles of sine-wave grating. Using conventional patches of grating, similarly high levels of summation (a factor of 1.7) have also been found across eyes (Baker, Meese, & Summers, 2007; Meese et al., 2006). This raises the question of whether vision can perform both of these summation processes together. For example, if the ‘white’ checks (Figure 1a) were presented to one eye and the ‘black’ checks (Figure 1b) to the other eye, then the binocular sum of those stimuli would be the full stimulus in Figure 1c. While this seems a likely candidate for area summation, it should be kept in mind that to achieve this, the visual system must integrate the contrast signal over different regions in the two eyes. Evidence either for or against this process would enhance our understanding of the organization of early human vision. 

As motivated above, our main aim here was to investigate conjoint summation of contrast over eyes and area. But for comparison, and by way of constraining our models, we also revisit the two forms of summation studied earlier. The logic and design of our three experiments is set out schematically in Figure 2. The shaded regions denote spatial luminance contrast (the target), and the open boxes denote potential locations for the contrast regions in a two-by-two factorial arrangement across eyes (vertical dimension) and area (horizontal dimension). To investigate conventional binocular summation the target was in the left eye alone, the right eye alone, or both eyes, but always the same spatial location ( Figure 2a; the ‘eyes’ experiment). To investigate monocular area summation, the target was in different or both spatial locations, but always the same eye ( Figure 2b; the ‘area’ experiment). To investigate summation across eyes and area together (our main experiment), the target was in either one spatial location in one eye, the other spatial location in the other eye, or both of these ( Figure 2c; the ‘eyes and area’ experiment). 

Figure 2

 

Schematic illustration of example stimulus arrangements for the three summation experiments here. The shaded (green) symbols provide an abstract representation of the target contrast regions. The open (blank) symbols are place markers in the figure and do not represent a form of stimulus in the experiments. The three columns denote the component pair and the compound condition for each of the three experiments (a, b, c). Within each quad of icons the different rows and columns denote stimuli presented to different eyes and different spatial locations, respectively. (In the experiments, location 1 is instantiated by ‘black’ checks and location 2 by ‘white’ checks.) (a) Binocular summation (‘eyes’ experiment). (b) Area summation (‘area’ experiment). (c) Summation across eyes and area (‘eyes and area’ experiment). (In the experiments, conditions were counterbalanced across eyes and area as summarized in Table 1.)

To interpret our results we performed an extensive analysis involving all possible (192) arrangements of the following model components: Linear summation or a MAX rule across eyes, linear summation or a MAX rule across area, linear and nonlinear contrast transductions, and (stochastic) Gaussian noise. Taken together, the most parsimonious interpretation of our results is that summation occurs across eyes and across area and that it is linear. The identification of a contrast nonlinearity between the two stages of summation allows us to conclude that binocular summation precedes area summation. These results also have implications for models of uncertainty and probability summation as we describe in the Discussion section. 

This extends out previous work (Meese et al., 2006; Meese & Summers, 2007) by (i) considering summation across eyes and space together and (ii) by providing a more thorough appraisal of possible models at detection threshold. 

Methods

Observers

Both of the authors (TSM and RJS) and a post-graduate student (SAW) served as observers. All three were psychophysically well practiced, wore their normal optical correction as appropriate, and had functional stereopsis. TSM and RJS had performed binocular experiments using similar stimuli previously (Meese & Summers, 2007). 

Equipment

Stimuli were viewed through Cambridge Research Systems (CRS) ferro-electric (FE-1) shutter goggles and displayed on a Clinton Monoray monitor with a frame rate of 120 Hz using a CRS ViSaGe stimulus generator. The mean luminance of the display viewed through the goggles was 16 cd/m ². The shutter goggles allowed different stimuli to be presented to the two eyes by interleaving across frames with a refresh rate of 60 Hz. Target contrast was controlled by look-up tables and gamma correction was performed to ensure contrast linearity. Observers sat at a viewing distance of 96 cm with their head in a chin and headrest fixating a dark square point (4.8 arcmin) placed in the center of the display throughout the experiment. The experiments were controlled by a PC. 

Stimuli

The three different types of stimulus are shown in Figure 1. The carrier was a horizontal sine-wave grating in sine phase with the center of the display and a spatial frequency of 2.5 c/deg. It was modulated by a circular raised cosine function with a central plateau of 8 deg, and a blurred boundary of 1 deg, giving a full-width at half-height of 9 deg ( Figure 1c). A ‘raised-plaid’ envelope was used to produce further modulation for two other stimuli. The plaid was the sum of two sine-wave grating components with orientations of ±45° and a spatial frequency of 0.5 c/deg, each with contrasts of 0.5. This gave minima and maxima of −1 and 1, respectively. The envelope was then ‘raised’ by adding 1 to each point and dividing by 2 throughout, giving minima and maxima of 0 and 1. Thus, the equation for the modulator was  

$env=\left(\begin{array}{c}1+\\ \\ \cos (2\pi [fx\cos \left(\theta \right)+fy\sin \left(\theta \right)]+\varphi )/2+\\ \\ \cos (2\pi [fx\cos (-\theta )+fy\sin (-\theta )]+\varphi )/2\end{array}\right)/2,$

(1)

where f is spatial frequency (=0.5 c/deg), θ is orientation (=45°), and ϕ is phase. There were two different phases of modulation: cosine phase ( ϕ = 0°; Figure 1a), and negative cosine phase ( ϕ = 180°; Figure 1b). These stimuli were given the nominal titles of ‘white’ and ‘black’ checks, respectively, as a reference to the magnitude of the modulator at the center of the display (unity and zero). Note that there are 7.07 cycles of carrier grating for every two checks (i.e., one cycle of a vertical cross-section through the envelope). 

Stimulus contrast is expressed as Michelson contrast in % of the carrier (i.e., C = 100[( L _max − L _min)/( L _max + L _min)]) or in dB re 1% (=20.log ₁₀( C)). 

Design

There were three experiments, referred to as ‘eyes,’ ‘area,’ and ‘eyes and area’. All three experiments measured contrast detection thresholds for each of a pair of components (single targets) and a compound made from the simultaneous presentation of the two components (dual targets). The experiments were fully counterbalanced across eyes and check ‘color’ (the nominal labels, ‘black’ or ‘white’). The set of conditions that formed a single repetition of each experiment is shown in Table 1 (deviations from this design are described in the Preliminary results section). A single experimental session (block) consisted of interleaved trials from either the four conditions on the left or right of Table 1 within each experiment and took about 20 minutes to complete. By treating the sensitivities for the different check ‘colors’ in each eye as equal (see Results section), a single experimental repetition delivered four independent estimates of summation (see caption of Table 1). Observers performed several repetitions of each experiment. The number ( N) of independent estimates of summation ratios contributing to the mean is indicated in the results figures as appropriate. 

Table 1

 

Stimulus conditions and design for the three experiments. Each experiment consisted of two blocks of four conditions. Within each block (session), trials were randomly interleaved from the four different conditions (two single and two dual target conditions). A single repetition of a single experiment involved sequential runs of the two blocks (in counterbalanced order). Each block contributed to the estimation of two summation ratios: the ratio (dB difference) of the detection thresholds ( α) measured for the conditions in the 1st and 3rd lines and the 2nd and 4th lines. To achieve a balanced design, the same condition appeared twice within a block in the ‘eyes and area’ experiment (lines 3 and 4) and across blocks in the ‘eyes’ and the ‘area’ experiments (lines 3 and 4).

Block 1		Block 2
Left eye	Right eye	Left eye	Right eye
‘Eyes’ experiment
Black checks	–	White checks	–
–	White checks	–	Black checks
Black checks	Black checks	White checks	White checks
White checks	White checks	Black checks	Black checks
‘Area’ experiment
Black checks	–	White checks	–
–	White checks	–	Black checks
Full stimulus	–	Full stimulus
–	Full stimulus		Full stimulus
‘Eyes and area’ experiment
Black checks	–	White checks	–
–	White checks	–	Black checks
Black checks	White checks	White checks	Black checks
Black checks	White checks	White checks	Black checks

Procedure

In most cases the procedure was as follows. The level of target contrast was selected using a method of constant stimuli with five different contrast levels, 3-dB spacing between them and 30 trials for each level. The levels used were determined from pilot work so that full psychometric functions could be measured in all conditions and experiments. We used a two-interval forced-choice (2IFC) procedure, where a null interval was blank (mean luminance) and the other interval contained the target. The onset of each 100 ms interval was indicated by an auditory tone and the duration between the two intervals was 400 ms. Observers were required to select the interval containing the target using one of two buttons to indicate their response. Correctness of response was provided by auditory feedback, and the computer selected the order of the intervals randomly. 

The exceptions to the procedure above were the ‘eyes’ and ‘eyes and area’ experiments for TSM. These were the same as above except that there were six contrast levels spaced 1.5 dB apart, and the duration between the 2IFC intervals was 300 ms. 

Psychometric functions (Ψ) were estimated by fitting Weibull functions to each individual run (180 or 150 trials) using psignifit (Wichmann & Hill, 2001a): 

$\Psi \left(c\right)=0.5+(0.5-\lambda )(1-\exp [-(C/\alpha {)}^{\beta }]),$

(2)

where C is the Michelson contrast of the carrier grating (in %). This equation has three free parameters. These are the ‘threshold’, α (the target contrast at 81.6% correct when λ = 0), psychometric slope (β), and the lapse-error rate, λ. The lapse-error rate is the proportion of trials in which the response is incorrect owing to finger errors and other observer miscues. One problem with the λ parameter is that if the observer's behavior is equivalent to λ = 0 (i.e., observers do not lapse), then this extra degree of freedom can lead to an oversteep estimation of β. To lessen the impact of this possibility, λ was capped at 0.01 in the fitting, as is appropriate for well-practiced observers (Wichmann & Hill, 2001b). 

To lessen the impact of outliers, estimates of slope were capped at β = 10 (see Wichmann & Hill, 2001b). Of the 252 psychometric functions that we measured, this occurred twice. 

Summation ratios (SR), or factors, are given by α _single/ α _dual, where α _single and α _dual are the thresholds for single and dual targets, respectively. Alternatively, they are expressed in decibels thus: SR = 20log ₁₀( α _single/ α _dual). 

Results

Preliminary results

Pilot work found that for TSM, contrast sensitivity to the ‘black’ and ‘white’ checks was equal in each eye (i.e., sensitivity was the same for all four check conditions), and this was borne out in the formal part of the study. 

For SAW, pilot work revealed that sensitivity was 3.3 dB higher for his left eye than for his right eye. In the formal experiments we compensated this by attenuating the left eye contrasts by 3.3 dB. This was successful in equating the sensitivity of the two eyes. Sensitivity to the two different check types was the same within eye for this observer. 

During data collection in the formal experiments, it became apparent that RJS was markedly more sensitive to the ‘white’ check stimulus in the right eye than the other three configurations (∼3 dB). For this reason, we restricted the summation analysis to the left eye in the ‘space’ experiment, to the ‘black’ checks in the ‘eyes’ experiment, and to ‘black’ checks in the left eye and ‘white’ checks in the right eye in the ‘space and eyes’ experiment. 

Experiment 1 (main): ‘Eyes and area’

The results for the ‘eyes and area’ experiment (our main experiment) are shown in Figure 3 for each of the three observers (different large black symbols). The coordinates of each data point indicate the mean level of summation (in dB) and the geometric mean of the slope of the psychometric function (Weibull β). The error bars (in this and all other plots) indicate the independent estimates of 95% confidence intervals for each of these parameters (compensated for the use of small samples). For all three observers, the level of summation is quite high (mean of 4.31 dB; a factor of 1.64) and the slope of the psychometric function is quite steep (geometric mean of β = 3.36; look ahead to Table 3 for details of individual results). The smaller points in Figure 3a (both black and red) are the predictions made by 62 different models, as we now describe (see 1 for implementation details). 

Figure 3

 

Results (summation ratios and slopes of the psychometric functions) from the ‘eyes and area’ experiment. Error bars show 95% confidence intervals. (a) The small points are predictions made by the 62 models outlined in the text. The four most successful models (see Results section for selection criteria) are highlighted in red and have slightly larger symbols than the rest (points ‘C’ and ‘D’ superimpose; see Table 1 for details). (b) The same results as in (a) but showing the loci of predictions for model A (solid curve) and model B (dashed curve) for a range of contrast transducer exponents p, where p = 1 to 3 from lower to upper parts of the curves. For the model involving the MAX operation over area (dashed curves), the predictions for the slope of the psychometric function were slightly different for the single (thin long dashed curve) and dual (thin dotted curve) targets. The thick medium dashed (red) curve between the two is their geometric mean.

We begin by restricting out interest to models with four elements: 1) binocular pooling ( POOL _eye), 2) spatial (area) pooling ( POOL _j), 3) contrast transduction f(), and 4) additive, Gaussian noise (+ G). We refer to these as the ‘four-element models’ and consider models containing a cascade of nonlinear transducers later. In principle, the four elements could be arranged in any order giving 4! = 24 feed-forward architectures. We first consider two types of pooling: linear summation (Σ) and a MAX rule (akin to probability summation when this follows noisy inputs; Pelli, 1985; Tyler & Chen, 2000). There are four possible combinations of the two types of pooling across eyes and area and we consider two forms of transducer (linear and accelerating). This gives 24 × 4 × 2 = 192 different model configurations. From these we removed the redundancies (e.g., if summation is MAX across space and MAX across eyes, adjacent pairings of these operations can follow in either order), and the equivalences due to Birdsall's theorem (see Lasley & Cohn, 1981), leaving 62 formally different models to be tested (though several produced very similar predictions). For 13 of these, the contrast transducer was linear (f(x) = x), whereas for the remaining 49 it was nonlinear (f(x) = x^2.4; Legge & Foley, 1980). 

For simplicity we restricted summation in the models to two full checks of the stimulus (one ‘black’ and one ‘white’). By symmetry, the predictions are the same for any integer multiple of such pairs of checks. Within each model, pooling occurred across eyes and space on every trial. The model simulations ( 1) produced predictions for the slope of the psychometric function ( β′) and the summation ratio (SR′). 

For some models there were small differences in the values of β′ predicted for the single and dual targets (the differences were always <1 for the models in Figure 3a). To avoid clutter in Figure 3a we plotted the average of these two estimates, though we return to this detail later (e.g., Figure 3b). 

For now we are looking for models that produced fair predictions of both the slope of the psychometric function and the level of summation. However, as it is straightforward to reduce the level of summation in the models by reducing the extent of pooling (Meese & Summers, 2007) we are not concerned, at this stage, if models overpredicted summation. The small filled circles (red and black) in Figure 3a show the 62 model predictions (some of which superimpose). From visual inspection, the models clearly fall into two groups. Those where the slopes of the psychometric functions are far too shallow (the steepest slope is β = 1.5) and those where the slopes of the psychometric functions are quite steep (the shallowest slope is β = 2.3, though β ≈ 3 is more typical). Within this second group, summation is too low in most cases. In fact, there are only four models for which summation is greater than the lower confidence limit of the observer with the weakest level of summation (SAW). These are shown by the slightly larger (red) points in Figure 3a, labeled A, B, C, and D (points C and D superimpose). In each of these models, the slopes of the psychometric functions are quite steep (e.g., β > 3). Only this group of four models¹ produced predictions consistent with the data according to the rejection criteria outlined above. The results in Figure 3a reject the other 58 models. In fact, from Figure 3a, model B is arguably marginal. However, it is the most successful model that involves the MAX operation over area, which is akin to spatial probability summation when it follows noise, as it does here (Tyler & Chen, 2000). And as models of spatial probability summation have a long history (e.g., Robson & Graham, 1981), we retain this model arrangement in our shortlist and further analyses below. 

A shortlist of four models

The four most successful models and their behaviors are listed in Table 2, and psychophysical behavior (averaged across the three observers) is shown for comparison in bold. 

Table 2

 

Architectures (top) of the 4 models short-listed for the quality of their predictions for the ‘eyes and area’ experiment. The first four rows indicate the order of the four model elements (pooling over eye [eye], pooling over area [ j], Gaussian noise [ G], and contrast transduction [ x ^p]), where the top row is the first element in the processing stream. Model summation ratios (SR′) and slopes of the psychometric function ( β′) are shown in the main body of the table and accompanied by the relevant psychophysical estimates in bold. Note that separate psychometric slopes for the single and dual conditions are shown for the models, whereas the average is shown for the psychophysical data.

Model code ( Figure 3a):	A	B	C	D
Model architecture (order of model elements)	Σ eye	Σ eye	Σ eye	MAX j
	x 2.4	x 2.4	Σ j	Σ eye
	G	G	x 2.4	x 2.4
	Σ j	MAX j	G	G
SR′ (dB) (model)	4.8	3.2	6.0	6.0
SR (dB) (data)	4.3	4.3	4.3	4.3
β′ single (model)	3.1	3.9	3.1	3.1
β′ dual (model)	3.2	3.0	3.2	3.2
β ′ average (data)	3.4	3.4	3.4	3.4

Model C ( Figure 3a, Table 2) involves perfect linear summation over eyes and area before nonlinear transduction and late noise. Although not rejected by our data here, it is inconsistent with results from studies of area summation where the size of a foveal grating has been varied (e.g., Foley et al., 2007; Meese, Hess, & Williams, 2005; Meese & Summers, 2007; Rovamo, Luntinen, & Näsänen, 1993; Summers & Meese, 2007). In those studies, summation was much less than the log–log (threshold vs. area) summation slope of −1 predicted by this linear model (model C). When retinal inhomogeneity is taken into account (Foley et al., 2007; Meese & Summers, 2007; Pointer & Hess, 1989) this flexes the summation curve to a concave shape that resembles the form of the data in those studies. But the summation slope remains too steep without the introduction of an accelerating (nonlinear) contrast transducer (Foley et al., 2007; Meese & Summers, 2007) or stimulus uncertainty (Pelli, 1985). Meese and Summers (2007) proposed that noise propagates from multiple sources before summing over area (cf. Campbell & Green, 1965 in the binocular domain), which also reduces the summation slope. In sum, the linear model (model C) of Table 2 is rejected on the grounds that it is inconsistent with other published results of conventional area summation experiments; it predicts too much summation, even when retinal inhomogeneity is included. 

Model D in Table 2 involves a MAX operation across area in each eye, followed by linear summation across eyes, then nonlinear transduction and late noise. This is a peculiar arrangement. The MAX operation over area means that the input to binocular summation arises from spatially isolated signals across the two eyes that are displaced by one check width. In the experiment here they have a disparity of √2 deg (equivalent to 3.5 carrier cycles), well outside the normal range of fusion and strong binocular summation (Howard, 2002; Rose, Blake, & Halpern, 1988). Furthermore, operating on mismatched features across eyes is a poor general strategy for binocular summation. All this makes model D seem unlikely. Moreover, this model is rejected outright by the results from our ‘area’ experiment, as we shall see below. This leaves only two viable models of our results (models A and B). 

In model A, summation is linear across eyes and area and nonlinear contrast transduction is placed between the two. The prediction sits fairly centrally among the three observers in Figure 3a and is in good agreement with the average results ( Table 2). 

Model B is the same as model A except that a MAX operation is used across area. The prediction is in fair agreement with the results, though summation is underestimated somewhat. ² This model ( Table 2) also predicts that the slope of the psychometric function should be less for the dual target than for the single target (see also Meese & Summers, 2007 and Tyler & Chen, 2000). We found no hint of this in the experiment (not shown) but the small change in model slope (from β′ = 3.9 to β′ = 3.0) is possibly too small to pick up reliably against the variability in psychophysical data (see confidence limits of β in Figure 3). 

In sum, only 4 out of 62 formally different models are (broadly) consistent with our data. Two of these (models C and D) are rejected by considerations outside the present work (see also the next section), leaving two viable arrangements (models A and B). These involve linear summation across eyes followed by a nonlinear transducer, additive noise, and either linear summation or a MAX operator across area. 

Model predictions for a range of transducer exponents

To investigate the behaviors of models A and B more thoroughly we reran the models for a range of contrast transducers. We used f( x) = x ^p for a range of exponents where p = 1 to 3. The loci of predictions are shown by the thick solid (model A) and dashed (model B) red curves in Figure 3b ( p = 1 to 3 from the bottom to the top of the curves). The data are the same as those in Figure 3a. The thin dashed gray curves show the different psychometric slopes ( β′) predicted by model B for the dual (short dashes) and the single (long dashes) targets. 

The results ( Figure 3b) for SAW sit between the two models and do little to decide between them. However, the results ( Figure 3b) for TSM and RJS clearly favor model A (Σ across area), and those of TSM far exceed the summation predicted by model B (MAX across area) for an appropriate slope of the psychometric function. 

Experiment 2: ‘Area’

The results for the ‘area’ experiment are shown in Figure 4. They are similar to the results for the ‘eyes and area’ experiment ( Figure 3), though average summation here is slightly greater (by 0.67 dB) than before. In Figure 4a, the predictions for model A and model B are shown as functions of p, as in Figure 3b. In fact, the curves are identical to those in Figure 3b because the first stage in both models is linear summation of contrast across eyes, which makes the two experiments equivalent for these models. Model A is clearly the superior model for all three observers. This extends the findings of Meese and Summers (2007) regarding area summation of binocular contrast to the monocular situation here. 

Figure 4

 

Results (summation ratios and slopes of the psychometric functions) from the ‘area’ experiment. Error bars show 95% confidence intervals. (a) The curves show the same model predictions as in Figure 3b plus that of model D (vertical locus, far left). (b) Predictions made by Minkowski summation. Different curves are for different summation exponents (γ) shown at the bottom of each curve. All model predictions are shown for a range of contrast transducer exponents, p, where p = 1 to 3 from lower to upper parts of the curves.

Also shown in Figure 4a is the prediction of Model D. In this model, the MAX response is found over space in each eye before summing across eyes. As there is signal in only one eye, this model cannot benefit from the linear summation across eyes, and so there is no summation (0 dB), regardless of the transducer exponent and slope of the psychometric function. As we advised earlier, this is a further reason to reject Model D. 

Minkowski summation

Another form of summation that has been used by modellers is that of Minkowski summation (Graham, 1989). This is often treated as a model of probability summation where the Minkowski exponent (γ) has a value of around 3 or 4 (Robson & Graham, 1981). However, more careful derivations for this form of summation have been developed from signal detection theory using the MAX operator (Pelli, 1985; Tyler & Chen, 2000), as in some of the models here. Nevertheless, Minkowski summation is convenient and continues to be widely used, and so for completeness we consider its application to the results here. 

We use a modified form of the equation suggested by Meese and Summers (2007), where the observer's internal response (resp_obs) to target contrast C is given by 

$res{p}_{obs}\left(C\right)={\left[\sum _{j=1:n}\right(res{p}_j\left(C\right){)}^{p\gamma }]}^{1/\gamma },$

(3)

where resp_j is the linear response to stimulus contrast C of the jth of n sensors in a spatial array (see 1 for details), p is the exponent of the contrast transducer, and γ is the Minkowski exponent. Equation 3 was solved for C assuming a criterion response of unity for single and dual targets to calculate model summation ratios. The slope of the psychometric function can be derived by fitting a Weibull function to the solutions to Equation 3 for a range of criterion response levels. However, here we used the direct approximation of Pelli (1987), where β = 1.247p. Note that γ has no effect on the slope of the psychometric function, as can easily be recognized by considering the case where n = 1 in Equation 3 and γ is cancelled. Equation 3 is usually (tacitly) used with p = 1. This produces the set of predictions at the lower ends of the three curves in Figure 4b, which are for three example values of γ (4, 2, and 1). Clearly, all are inadequate in terms of the slope of the psychometric function. Increasing the value of p brings the predictions much closer to the data, the best situation shown being for γ = 1 (solid red curve). In fact, that version of Equation 3 is the deterministic equivalent of model A, the very slight differences between the (solid red) curves (in Figures 4a and 4b) owing to the different approximations used in the two methods of analysis. 

For this experiment at least, even the modified version of Minkowski summation ( Equation 3) has nothing to offer over model A (see also Meese & Summers, 2007). Furthermore, using Equation 3 with γ = 3 or 4, as in the widely used approximation to probability summation, is clearly inadequate. 

Experiment 3: ‘Eyes’

The results of the ‘eyes’ experiment are shown in Figure 5. They are similar to the results from the ‘eyes and area’ experiment ( Figure 3), though average summation here is a little greater (by 0.72 dB) than before. 

Figure 5

 

Results (summation ratios and slopes of the psychometric functions) from the ‘eyes’ experiment. Error bars show 95% confidence intervals. (a) The solid vertical red line shows the predictions for the same two models as in Figures 3b and 4a, which in this case superimpose. (b) Predictions for alternative models of binocular summation where the transducer (here referred to as exponent m) precedes either linear summation (solid blue curve) or a MAX operation (medium dashed blue curve). These processes are then followed by area summation and additive noise. Predictions are for a range of contrast transducer exponents, m, where m = 1 to 3 from the lower to upper parts of the curves. The thin dotted and dashed gray curves are as for Figure 3b.

The solid (red) curve in Figure 5a is the prediction made by both model A and model B. Both involve linear summation of contrasts across eyes before nonlinear contrast transduction, and therefore predict summation of 6 dB (a factor of 2) regardless of the value of p and the slope of the psychometric function. This slightly overestimates the level of binocular summation found in this and other experiments (e.g., Baker et al., 2007; Meese et al., 2006). 

For completeness, Figure 5b shows predictions for alternative models (rejected already by the analysis of Figure 3) where a single stage of contrast transduction (we now refer to the exponent as m) precedes binocular pooling and is then followed by spatial pooling and additive noise. When binocular pooling is a MAX operator (thick dashed blue curve) the model fails to reach the requisite levels of summation, regardless of the slope of the psychometric function, illustrating the resounding failure of that model (Meese et al., 2006). When the pooling is linear (solid blue curve), the requisite levels of summation can be achieved, but the slopes of the psychometric functions are far too shallow. In fact, the ‘eyes’ result of Figure 5 implies at least two stages of nonlinear relation between stimulus contrast and response (Baker et al., 2007). We describe and extend this idea in the next section. 

A three-exponent cascade model of summation

To reduce the amount of binocular summation in the model (from that in Figure 5a), we inserted an additional nonlinear contrast transducer before binocular summation (with exponent m). This is shown in the schematic outline in Figure 6. With this arrangement it is possible to calculate the values of m and p that are needed to exactly fit the summation ratios found in the ‘area’ and the ‘eyes’ experiments ( Table 3, middle; see 1 for details). However, this arrangement does not produce good predictions for the slopes of the psychometric functions. For example, for TSM and SAW the cascade of fairly low exponents leads to an underestimation of β. (Details are not shown, but the overall exponent is equal to the product of those in the cascade, and from Pelli (1987), this is multiplied by 1.247 to predict β.) This problem was addressed by introducing a third and final exponent (u) placed after the final stage of summation but before the limiting noise (Table 3 and Figure 6). Thus, to summarize, m is the exponent needed to fit binocular summation, mp is the exponent needed to fit area summation, and mpu is the overall exponent needed to fit the (average) slope of the psychometric function. Finally, the limiting noise must be late in order that u can affect performance, owing to Birdsall's theorem. 

Figure 6

 

Cascade model with two stages of summation (eyes and space) and three exponents, m, p, and u, representing nonlinear relations between stimulus contrast and response. L and R denote local contrasts in the left and right eyes respectively, and the subscripts denote different corresponding points across the retinae. For the behaviors of the deterministic models in Table 3, the limiting additive noise was placed immediately before the decision maker. The decision maker chooses the 2IFC interval that produces the greatest response.

Table 3

 

Summary of experimental results for each observer and their average (top), and behaviors of two cascade models involving two stages of summation and three stages of nonlinear contrast transduction (middle and bottom). The parameter P is the effective overall exponent and is estimated from the psychophysical results using Pelli's (1987) approximation. This is the (average) estimate of the slope of the psychometric function (β―) divided by 1.247. In the models, the overall exponent is given by the product mpu. The exponents m and p were estimated from the SRs in the ‘eyes’ and ‘area’ experiments respectively, leaving u to be set by P. The equivalent uncertainty (U) is described in the Discussion section. No degrees of freedom remained for predicting the SRs in the (main) ‘eyes and area’ experiment. Note that the model parameters in the ‘average’ column are not the averages of those in the other three columns, but the parameters fit to the data of the average observer.

Observer	TSM	RJS	SAW	Average
Experimental results
Eyes SR	4.95	5.87	4.26	5.03
Area SR	5.10	4.83	5.00	4.98
Eyes and area SR	4.40	4.88	3.64	4.31
Eyes β	4.1	2.5	3.6	3.3
Area β	3.8	2.6	3.6	3.3
Eyes and area β	4.0	2.4	4.0	3.6
β ―	3.9	2.5	3.7	3.3
P = ( β ―/1.247)	3.2	2.0	3.0	2.7
Model: Summation across eyes then area
m	1.22	1.03	1.41	1.20
p	1.60	2.26	1.47	1.76
mp	1.95	2.32	2.07	2.11
u = P/( mp)	1.62	0.86	1.45	1.26
Equivalent uncertainty ( U)	14	N/A	7	3
Predicted SR for ‘eyes and area’	4.27	4.69	3.70	4.20
Model: Summation across area then eyes
m	1.95	2.32	2.08	2.11
p	0.62	0.44	0.68	0.57
mp	1.22	1.03	1.41	1.20
u = P/( mp)	2.60	1.94	2.13	2.22
Equivalent uncertainty ( U)	757	51	111	164
Predicted SR for ‘eyes and area’	4.95	5.87	4.26	5.02

With all model parameters set, predictions were then derived for the ‘eyes and area’ experiment. These were very good, as shown in Figure 7 by the close proximity between the large solid (black) symbols (human data) and the small solid (red) symbols (model). 

Figure 7

 

Summation ratios and slopes of the psychometric functions for the ‘eyes and area’ experiment (top of Table 3), replotted from Figure 3. Error bars show 95% confidence intervals. For the average of the three observers (solid black square) this was calculated using a bootstrapping technique. The small solid (red) and open (blue) symbols are the cascade model predictions (no free parameters) for each observer and the average observer (different shapes). The parameter values for these models are shown in the middle and bottom parts of Table 3.

The order of summation

For the original four-element models ( Figure 3a) it was important that binocular summation was placed before area summation ( Table 2). When the order was reversed (in models A and B), the predicted level of summation for the ‘eyes and area’ experiment was insufficient (<2.7 dB) for each form of summation (MAX and linear). But with the limitation of a single transducer relaxed above, do the results still constrain the order of summation? To answer this we considered a model similar to that in Figure 6, but with the two summation stages exchanged (the exponent order was unchanged). This produced different values for each of the exponents, m, p, and u ( Table 3, bottom) after fitting to the ‘eyes’ and ‘area’ experiments as before. Most notably, p is compressive (<1) for this arrangement. More importantly, however, this model predicted too much summation for the ‘eyes and area’ experiment in all cases (open small [blue] symbols in Figure 7, and Table 3, bottom), causing it to be rejected in favor of the arrangement in Figure 6. 

Discussion

Linear summation of contrast takes place across eyes before space

The summation results here provide good evidence for a cascade of contrast pooling operations at detection threshold. In experiment 1 (‘eyes and area’), summation was found when the dual signal regions were placed in different spatial locations in different eyes, indicating neuronal convergence from different eyes and different retinal locations. We tested 62 formally different four-element models of the process. Only one of these survived our detailed consideration of the ‘eyes and area’ experiment and other results elsewhere. This model (model A) asserted a strict order to the pooling: linear summation across eyes takes place before nonlinear contrast transduction and linear area summation. However, to provide good accounts of the summation ratios and slopes of the psychometric functions for all three of our experiments (‘eyes,’ ‘area,’ and ‘eyes and area’) it was necessary to introduce a cascade of three transducers (with exponents m, p, and u; Figure 6). Even so, the overall analysis clearly favored a scheme in which binocular summation is placed before area summation, rather than the other way around ( Figure 7 and Table 3). 

In a very different type of study, Mansouri, Hess, Allen, and Dakin (2005) came to the conclusion that binocular summation precedes the pooling of multiple orientation signals across space. Hess and Field (1995) found that spatial contour integration arises after disparity processing, and Huang, Hess, and Dakin (2006) came to a similar conclusion. All this suggests the possibility of a fairly general scheme in which higher order spatial computations follow the early representation of image data combined from the two eyes. 

Probability summation and the MAX operator

Our analysis revealed shortfalls for models of probability summation, whether implemented by a Minkowski metric ( Figure 4b) or a MAX operator (following noise) in a four-element model ( Figures 3b, 4a, and 5b). However, with the limit to four model elements relaxed, might an implementation emerge by which the spatial MAX operator could survive? The earlier failure of this operation (model B) in Figure 4a owes to its inability to predict the slope of the psychometric function. But suppose that a MAX operation is placed at the area summation stage in Figure 6 (replacing the linear sum) following low values of m and p (to fit the high levels of summation; Figures 4 and 5) and that additive noise is placed after the transducer p but before the spatial MAX operator (as in model B; Table 2). Now a high value of u and subsequent late performance limiting noise (as in Figure 6) could be included to increase the slope of the psychometric function without influencing the level of summation. 

Note that this arrangement (not shown) is rather different from the usual conceptualization of the MAX operator, which is placed at the end of the signal processing chain, just before the decision variable (Pelli, 1985). With that arrangement, the MAX can be treated as an operation performed by the decision-making process. For the arrangement offered here, the placement of model elements (a transducer and performance limiting, additive noise) between the MAX operation and the decision variable suggests a MAX operation that forms part of the sensory pooling. In any case, this is the only arrangement (that is limited by additive noise) in which the spatial MAX operator can survive in a model of spatial summation for the type of stimuli used here. 

Uncertainty

The models involving MAX operations here include an implicit component of extrinsic uncertainty for the single target conditions (consistent with the intermixing of single and dual conditions in the experiments). One of the effects of this type of uncertainty is to increase the slope of the (model) psychometric function (Pelli, 1985; Tyler & Chen, 2000; see also Table 2, model B, second and third rows from bottom). Nevertheless, for the models (Figure 3a) with a linear transducer, the model slopes were much shallower than those measured in the experiment. In general, this shortcoming was corrected in the models by including one or more accelerating contrast transducers that further increased the slope of the psychometric function. But another approach is to increase the background level of intrinsic uncertainty (Pelli, 1985). However, this does not help reinstate the spatial MAX operator because increasing uncertainty also decreases the level of summation, in a very similar way to the nonlinear transducer in Figure 4a (Summers & Meese, 2007; Tyler & Chen, 2000). 

In general, when d′ is in the range measured in detection experiments, an accelerating contrast transducer followed by noise can be replaced with a MAX rule that operates on noisy input lines, where the level of uncertainty, U, is the number of input lines that are monitored by the observer (and the signal is carried by a single input line). The amount of uncertainty needed grows exponentially with the transducer exponent for which it is intended to replace (Pelli, 1985). In fact, there are good reasons to replace the output stage in Figure 6 (exponent u and late noise) with this arrangement, as depicted in Figure 8. Because the area summation stage is linear the limiting noise can be moved to the left-hand side of that stage, placing it earlier than in Figure 6. This arrangement was an important part of the Meese and Summers (2007) model of area summation for the situation where a central patch of grating is grown in size and there is no extrinsic uncertainty. In that model, the region of linear area summation was matched to the size of the target, following retinal inhomogeneity, a nonlinear transducer, and additive noise. This predicted the moderate levels of summation found in the experiment owing partly to the growth of noise with signal area. Thus, we envisage that the area summation region in Figure 8 can be matched to the target diameter, at least up to some range (Meese & Summers, 2007; Summers & Meese, 2007). Whether this involves a flexible mechanism of variable size, or the selection of an appropriate sized pooling mechanism within a discrete set, is unclear. 

Figure 8

 

Alternative model to that in Figure 6. This model incorporates intrinsic uncertainty instead of the output transducer u. The observer monitors a total of U mechanisms (A₁: A_U) only one of which is relevant. The irrelevant mechanisms have comparable levels of noise, as might occur if they performed area summation over irrelevant carrier orientations, spatial frequencies, and so forth. As in Figure 6, the decision maker chooses the 2IFC interval that produces the greatest response.

The levels of uncertainty ( U) needed for the model in Figure 8 were estimated using Pelli's (1985) approximation (his equation 5.4) and are reported in the middle part of Table 3. Note that they are quite modest (because u was quite low), as might be expected for highly trained observers. (For completeness, U is also reported in Table 3 for the less successful model in which binocular summation follows area summation. Those values are much higher.) 

Summary and conclusions

For more than 25 years, studies of spatial vision have been dominated by the view that spatial contrast pooling is weak at detection threshold (e.g., probability summation). This view was first challenged in our recent companion paper (Meese & Summers, 2007), where we concluded that a signal combination strategy takes place across several grating cycles at threshold and above. In the binocular domain, Meese et al. (2006) recently concluded that contrast summation across eyes is greater than the factor of √2 (3 dB, or quadratic summation) that has often been supposed (Campbell & Green, 1965). The current study has extended the inquiry by combining these two dimensions to investigate pooling of signals presented to different eyes and different spatial (retinal) locations. Our results confirm that the pooling strategy involves stimulus combination across eyes and area together. Our most successful model indicates a strict order to the pooling: area pooling follows binocular pooling. A model incorporating a spatial MAX operator might survive in an early sensory context if it is followed by subsequent stages of contrast response nonlinearity and performance limiting noise. However, our preferred model (owing to its relative simplicity) involves linear summation at both pooling stages with moderately accelerating contrast transduction before each stage, performance-limiting noise before area summation, and a late stage of signal uncertainty (Figure 8). 

Appendix A

Four-element model architectures

The models for which predictions are shown in Figure 3a have four components: 1) binocular pooling ( POOL _eye), 2) area pooling ( POOL _j), 3) contrast transduction f() and 4) additive, Gaussian noise (+ G). We used two types of pooling: linear summation and a MAX rule, giving POOL( x ₁.. x _n) = Σ( x ₁.. x _n) or POOL( x ₁.. x _n) = MAX( x ₁.. x _n), respectively, where x _i is the contrast response at the appropriate stage in the model (see below). For Figure 3a there were two types of transducer. For the linear transducer, f( x) = x, and for the nonlinear transducer, f( x) = x ^2.4 (Legge & Foley, 1980). (Owing to the simplification below, x is always ≥0.) For the model curves in Figures 3b, 4a and 5b, the exponent in this expression was varied over the range 1.0 to 3.0 in steps of 0.2. 

Monte Carlo simulations

We assumed that first-order linear filters provided sufficiently dense sampling over space (and phase) to treat the contrast envelope as the analytic signal. We performed the analysis by considering the responses across two-dimensional arrays of filter elements (sensors, j) over the region of two whole checks, one ‘black’ and one ‘white’ ( j = 1 to n, where n = 31 × 63 = 1,953, though this figure is not critical). The left-eye and right-eye linear sensor responses to unit contrast in the ‘eyes and area’ experiment are shown in Figure A1, and are given by L _j and R _j. The linear responses to any stimulus contrast (in %) are given by the product of these terms with target contrast C. 

Figure A1

 

Monocular sensor-response distributions to a unit contrast stimulus of different check phase (‘black’ or ‘white’) in the two eyes, over two full checks for a dual target (‘eyes and area’ experiment). The linear sum of the two distributions is a unit response for all sensors. The sensor arrays for the ‘eyes’ and ‘area’ experiments are as described in the text.

In all experiments, the single target produced the response distribution shown by that in the left of Figure A1. The sensor responses in the right eye were zero. In the ‘eyes’ experiment the dual target produced a response distribution in the right eye that was the same as that in the left eye. In the ‘area’ experiment, the dual target was a unit response for all sensors in the left eye and the responses of the right eye sensors were zero. 

We added zero-mean, unit-variance ³ Gaussian noise ( G) drawn independently on each interval of each simulated trial for each transmission line at the stage appropriate for the injection of noise for each model. 

The four model components ( POOL _eye(), POOL _j(), f(), and + G) were combined in each of the orders appropriate for the various model architectures under test. For example, the sequence: POOL _eye, f( x), G, POOL _j, is given by  

$\begin{array}{c}resp\left(C\right)=\\ \\ POO{L}_j\left\{f\right(POO{L}_{eye}\left[L_j\right(C),R_j(C\left)\right])+G_j\},\end{array}$

(A1)

where C is target contrast. Note that pooling took place across eyes and all spatial locations in both 2IFC intervals in each experiment. 

On each simulated 2IFC trial, C appeared in only one interval. The simulated observer selected the interval that produced the largest response using the appropriate model equation (e.g., Equation A1). If the interval contained the target, then the decision was correct, otherwise it was incorrect. The simulations were performed for a wide range of values of C in 0.5 dB steps with 2000 trials at each level (a simulated method of constant stimuli). This was done with single and dual increments of C. The simulated results were fitted with Weibull functions ( Equation 2) to produce predictions for the slopes of the psychometric functions ( β′) and thresholds at 81.6% correct ( α′). Model summation ratios (SR′) were given by: 20log ₁₀( α′ _single/ α′ _dual). 

Confidence intervals for our stochastic models were calculated using a bootstrapping method and were negligible (i.e., 95% confidence intervals typically fell within ±0.2 dB of the reported summation ratio and ±0.1 of the reported β). 

In Figures 3b, 4a, and 5b, the model predictions were fitted with third-order polynomials to smooth out very minor deviations that arose from their stochastic origins. 

Minkowski summation

The model curves in Figure 4b were derived by applying Equation 3 (main body) to the sensor arrays (see Figure A1) as appropriate for the area experiment. The model is deterministic (stochastic noise was unnecessary) and Equation 3 was solved directly for C, assuming unit response at detection threshold, to calculate summation ratios for the single and dual targets. The slope of the psychometric function ( β) was estimated using Pelli's (1987) approximation where β = 1.247p, and p is the (overall) exponent of the contrast transducer. 

Three-exponent cascade models

Two versions of this model were implemented using sensor arrays similar to those in Figure A1, as appropriate for each experiment and condition. In one version, the sequence of model elements was as shown in Figure 6, for the other the order of binocular and area summation was reversed. Both involved three stages of contrast transduction with exponents, m, p, and u in that order. Both also involved linear summation at the two pooling stages (see Table 3 and Figure 6) and were therefore implemented deterministically, as for the Minkowski summation. For the version in which binocular summation preceded area summation ( Table 3, middle), the exponent m was determined by analytic solution of the model for the empirical summation ratios (SR) in the ‘eyes’ experiment. The exponent product mp was then solved numerically for the empirical SR in the ‘area’ experiment. Finally, the exponent product mpu was solved analytically using Pelli's (1987) approximation: mpu =

$\bar{\beta }$

/1.247, where

$\bar{\beta }$

is the Weibull slope parameter of the psychometric function averaged across the three experiments (to achieve the best possible empirical estimate). 

With all model parameters fixed, the predictions for the ‘eyes and area’ experiment were determined numerically by solving the model equation for C for the single and dual targets in that experiment. The slopes of the model psychometric functions were determined by the fitting of m, p, and u (described above) and were equal to the average empirical estimate,

$\bar{\beta }$

. 

For the model in which binocular summation followed area summation ( Table 3, bottom), the procedure was the same as above, except that the exponent m was determined by numerical solution for the SR from the ‘area’ experiment and mp was then solved analytically from the SR in the ‘eyes’ experiment. 

Acknowledgments

This work was supported by grants from the Engineering and Physical Sciences Research Council (GR/S74515/01) and the Wellcome Trust (069881/Z/02/Z). 

Commercial relationships: none. 

Corresponding author: Tim S. Meese. 

Email: [e-mail]. 

Address: School of Life and Health Sciences, Aston University, Birmingham, B4 7ET, UK. 

Footnotes

Footnotes

1  A fifth model fell just outside the lower summation bounds in Figure 3a. For this model, the four elements were arranged in the following order: MAX over eyes, Σ over area, transducer p = 2.4, additive noise. As for the other models, summation could be increased by reducing the value of p, but this also reduced the slope of the psychometric function. We could find no adequate transducer for this arrangement.

Footnotes

2  Eagle-eyed readers might observe what appears to be a discrepancy in the analysis of two of our studies. In the work here the summation ratio (SR) predicted for model B (spatial MAX operation after a nonlinear exponent of 2.4) is 3.2 dB ( Table 2), whereas in Meese and Summers (2007), the SR predicted for the same model is 3.5 dB (Table 1 in that study). The reason for the slight difference is that the SRs were calculated using different points on the psychometric functions (P81.6 here and P75 in Meese & Summers). If the slopes of the model psychometric functions were the same in the single and dual conditions the threshold criterion would not matter, but for the MAX operator the psychometric slope is slightly steeper for the single target than the dual target because of uncertainty (Tyler & Chen, 2000). This causes the level of SR to fall slightly as the calculations are performed at progressively higher points on the psychometric functions.

Footnotes

3  The choice of unit variance noise was not critical. Indeed, the individual differences in contrast sensitivity (not shown) might be attributed to different levels of noise across observers. However, in this study we are concerned with ratios of sensitivities (summation ratios), for which we need assume only that the variance of the noise is the same for the two conditions contributing to the ratio (within observer). So long as this assumption holds, the individual differences in SRs cannot be attributed to individual differences in the level of internal noise.