Noise reveals visual mechanisms of detection and discrimination

Abstract

When performance is limited by stochastically defined masks, (psychophysical) reverse correlation has proven to be an especially efficient tool for estimating the templates used by detection and discrimination mechanisms. Here I describe a maximum-likelihood approach to quantifying the significance of differences between estimates of template. Four methodologically related experiments illustrate the versatility of reverse correlation. Experiment 1 shows significant differences between the templates used by different observers when detecting a bright Gaussian blob. The results of Experiment 2 are consistent with observers not using information about the phase of a parafoveal wavelet when detecting it. Experiments 3 and 4 reveal not only the templates used by detection mechanisms but also aspects of their response functions. Both results are consistent with a sensory threshold. Experiment 3 shows that 2-alternative forced-choice detection errors are caused when the target’s effective contrast is reduced, not when the mask looks more like the expected target+mask than the actual target+mask. Experiment 4 suggests that observers use optimally tuned detection templates for orientation discrimination.

General Background

Terminology

Most contemporary models of visual function employ mechanisms that mimic the behavior of individual neurones (for a review, see Graham, 1989). Analogous to a neurone’s receptive field, each mechanism has a template: the greater the match between template and stimulus, the greater the mechanism’s output. Note that a single mechanism may reflect the activity of just one or countless neurones. 

To measure visual function, human observers are asked to respond to various visual stimuli. Each response can usually be classified as correct or incorrect. Threshold stimuli are those that elicit a criterion response accuracy. For example, in a detection task, where response accuracy increases with stimulus contrast, threshold may refer to that contrast for which 75% of the responses are correct. 

Templates Revealed by Conventional Masking Experiments

Traditionally, there were three ways to learn about templates: summation, adaptation, and conventional masking experiments. All three paradigms involve comparisons between thresholds. Within the summation paradigm, comparisons are made between threshold for a single target pattern and threshold for a number of target patterns. Within the adaptation paradigm, comparisons are made between threshold for the target alone and threshold for that same target when displayed after an adapting stimulus. 

More germane to the current discussion are conventional masking experiments, where template properties can be deduced by systematically varying certain aspects of a masking stimulus and examining the effect upon threshold for a simultaneously presented target. Many researchers have used this method to obtain spatial-frequency descriptions of various templates (Henning, Hertz, & Hinton, 1981; Losada & Mullen, 1995; Pantle & Sekuler, 1968; Pelli, 1980; Solomon & Pelli, 1994; Stromeyer & Julesz, 1972). These conventional masking experiments often require many data, particularly if they are to avoid the problem of off-frequency looking (Losada & Mullen, 1995; Pelli, 1980; Perkins & Landy, 1991; Solomon, 2000). Another problem with this technique is that attempts to deduce a template’s spatial layout have been confounded by apparent interactions between detection mechanisms (Polat & Sagi, 1993, 1994). 

Receptive Fields Revealed by Reverse Correlation

On the other hand, many of today’s physiologists learn about receptive fields by studying the effects of samples of visual noise on neural output (for a review, see Marmarelis & Marmarelis, 1978). By averaging together all of the samples preceding action potentials by varying durations, the entire spatio-temporal structure of a neurone’s receptive field can be revealed. 

Templates Revealed by Reverse Correlation

Ahumada (1996) was the first to adapt this reverse-correlation approach for visual psychophysics. Instead of using a different mask every session and recording its effect upon threshold, he used a different mask every trial and recorded its effect upon response. Subsequently, Watson & Rosenholtz, 1997; Watson, 1998; Abbey, Eckstein, & Bochud, 1999; Ahumada & Beard, 1999; Knoblauch, Thomas, & D’Zmura, 1999; Neri, Parker, & Blakemore, 1999; Beard & Ahumada, 2000; Gold, Murray, Bennett, & Sekuler, 2000; Abbey & Eckstein, 2000 and Knoblauch & Yssaad-Fesselier, 2000 used similar techniques. Ringach (1998) described a related but dissimilar technique. By adding together (pixel-by-pixel) all of the masks eliciting correct responses and subtracting off all of the masks eliciting incorrect responses, Ahumada produced a picture of the template for the mechanism subserving vernier discrimination. 

Theory

Signal-Detection Theory for Noise Masking

Response rule for 2AFC

Signal-detection theories (SDTs), first applied to vision by Tanner and Swets (1954; see Green & Swets, 1966 for a more complete history), remain the most popular account of those conventional masking experiments that use noise masks. Equation 1 describes the simplest SDT when applied to the 2-alternative forced-choice (2AFC) procedure. On each trial i of this procedure, observers must select the one of two displays of visual noise that also contains a target pattern. In Equation 1, the target t and both samples of noise n_i1 and n_i2 are represented by vectors. For simplicity, assume that both target and noise are gray-scale (as opposed to color) images. In that case, each entry in each vector describes the intensity of a particular pixel. The template, represented by the vector w, describes the detection mechanism’s relative sensitivity to each pixel.  

 

(1)

specifies the rule for a correct response: the match (i.e., the inner product) between the target-present display and the template must exceed the match between the target-absent display and the template. To account for the fact that observers do not always respond in the same way to the same stimuli, each match is thought to be perturbed by internal noise. η_i1 and η_i2 represent two samples of that noise. 

Response rule for Y/N

A simple modification of Equation 1 yields a description of the rule for a “Yes, I see it” response in the yes/no (Y/N) procedure:  

 

(2)

 

In Equation 2, the vector s_i represents the display on trial i, which may or may not contain the target. c is an arbitrary constant called the internal criterion. Each of these equations makes a prediction that has been confirmed by numerous conventional experiments (Pelli, 1990): threshold contrast increases linearly with the contrast of the noise mask. 

Template Estimation

The standard analysis

Abbey et al. (1999) proved that Equation 1 implies the sum of the noise masks from selected displays minus the sum of the noise masks from the other displays yields an unbiased estimate of the template. A similar theorem is proved in “1”: Equation 2 implies the sum of the noise masks eliciting “yes” responses minus the sum of the noise masks eliciting “no” responses (to either target-present or target-absent trials) yields an unbiased estimate of the template. These are examples of what I call the standard analytical technique for psychophysical reverse correlation. In all of the psychophysical reverse-correlation studies cited above, template estimates were derived by simply adding and/or subtracting individual noise masks, although in some of these studies, samples eliciting “yes” responses and samples eliciting “no” responses were normalized by their number (i.e., they were averaged) prior to subtraction. 

Multiple regression

Equation 2 suggests an alternative analysis based upon multiple regression. For any constant b and any series of vectors s_i and constants r_i, i=1,…,N, multiple regression is the conventional statistical technique that finds that w which minimises in  

 

(3)

(Mood, Graybill, & Boes, 1974). 

Recognizing the similarity between Equations 2 and 3, Ahumada and Lovell (1970) presented auditory noise and asked observers to rate how confident they were that a target tone was present. If these ratings indeed reflect multiple internal criteria, then multiple regression is the most efficient way to reveal the template for the tone-detection mechanism. 

Maximum likelihood

Several psychophysical reverse-correlation studies report systematic differences between the template an ideal mechanism would use (for target detection in white noise, this is simply the target itself) and the templates used by visual mechanisms. However, the significance of these differences is rarely quantified. In several of the experiments reported below, I use a maximum-likelihood analysis (MLA) both to derive template estimates and to quantify their difference from the ideal. (For comparison, I also use the standard analytical technique.) This MLA assumes that the internal noise is zero-mean Gaussian, with standard deviation σ. 

The first step of the MLA is to derive the formula for response likelihood. The assumption of Gaussian internal noise, when coupled with the response rule in Equation 1, implies the following formula for the likelihood of a correct response on any trial i:  

 

(4)

 

where Φ(x) is the standard normal CDF (cumulative density function). The likelihood of an incorrect response is 1 − ψ_i. Similarly, the assumption of Gaussian internal noise, when coupled with the response rule in Equation 2, implies the following formula for the likelihood of a “yes” response on any trial i:  

 

(5)

 

The likelihood of a “no” response is 1 − ψ_i. For clarity, any parameters appearing in the likelihood formula will be called response parameters. Note that Equation 4 has one response parameter: σ. Equation 5 has two: σ and c. 

For any given template w, one needs only to assign values to the response parameters in order to calculate the joint likelihood of all responses collected. For example, if the responses to trials 1, 2, and 3 were “yes,” “yes,” and “no,” respectively, then the joint likelihood of these responses would be ψ₁ψ₂(1− ψ₃). The second step of the MLA is to find those values for the response parameters that maximize the joint likelihood of all responses collected by each observer, given no difference between the ideal and visual templates, (i.e., for target detection in white noise) w = t. I use Mathematica’s FindMinimum routine (Wolfram, 1999) to find these parameter values. 

Templates themselves are specified by another set of parameters (the template parameters). With no constraints, the number of parameters required to completely specify a template will be equal to the number of elements in the vector that represents it. In the MLAs described below, I make certain assumptions about the form of visual templates, which reduce the number of parameters required for their specification. If, for example, the ideal template is a Gabor pattern, then it is not unreasonable to assume that the visual template is also a Gabor pattern, which can be fully specified by 6 parameters: frequency, orientation, phase, spread, horizontal position, and vertical position. (Because the standard deviation of internal noise σ is a free parameter, template amplitude can be assigned any arbitrary value.) In step three of the MLA, the maximum joint likelihood of all responses is again determined, this time with some subset of the template parameters allowed to vary from their ideal values along with the response parameters. 

Because the ideal template satisfies the constraints upon the visual template, twice the natural logarithm of the ratio between their maximum likelihoods (found in steps two and three, respectively) should follow the χ² distribution, with degrees-of-freedom equal to the number of freely varying template parameters in step three (Mood et al., 1974). The significance of the difference between ideal and visual templates can therefore be quantified. 

Theories for Pattern Masking

Sensory threshold

Threshold contrast does not increase linearly with the contrast of masks that are not composed of visual noise (Legge, 1981), and SDT must be modified accordingly. In fact, for some of these pattern masks, threshold actually decreases as their contrast is increased from zero (Nachmias & Sansbury, 1974). A simple modification that will allow SDT to account for this facilitation is a sensory threshold. Not to be confused with the performance thresholds discussed above, a sensory threshold serves to attenuate the output of a detection mechanism when its template is poorly matched by the visual stimulus. With this modification, the rule for a correct response in the 2AFC procedure becomes  

 

(6)

and the rule for a “yes” response in the Y/N procedure becomes  

 

(7)

where f(x) increases faster than x. The simplest such function is a “hard” threshold (see “2” for details):  

 

(8)

Note that both displays in the 2AFC procedure now contain the same mask n_i. 

Similarity

Consider a 2AFC noise-masking experiment in which a different sample of noise is used on every trial, but within each trial, both displays contain the same sample. Conventional explanations of performance in a twinned-noise experiment such as this typically posit no direct relationship between any particular sample of noise and response accuracy (Beard & Ahumada, 1999; Burgess & Colborne, 1988). The one exception is Watson, Borthwik, and Taylor’s (1997) similarity model in which the rule for a correct response is  

 

(9)

where ‖·‖ denotes vector length. It follows that the samples least likely to cause errors are those that are least similar to the template. The sum of the samples eliciting errors (or the inverse of the sum of the other samples) e should therefore be positively correlated with the template (i.e., w^te>0). Thus, if the template has any similarity to the target (i.e., w^tt>0), Then t^te>0. 

On the other hand, if the sensory-threshold theory (Equation 6) were correct, then in a twinned-noise experiment those samples that are least likely to cause errors are those that are most similar to the target. Thus, t^te should be less than zero. Simulations of these models are shown in conjunction with Experiment 3, which corroborates the prediction of sensory-threshold theory. 

Uncertainty

All of the analyses described above assume that the observer will use the same mechanism (with template w) on every trial of a particular experiment. For many experiments, this assumption may not be correct. Indeed, one popular model of detection posits that some responses are based on the outputs of mechanisms that are completely insensitive to the target (Pelli, 1985). Support for this theory can be demonstrated (see Ahumada & Beard, 1999 and below) using a Y/N procedure. Hits (i.e., “yes” responses to targets of non-zero contrast) are likely to be caused by stimulation of mechanisms that actually are sensitive to the target, whereas false alarms (i.e., “yes” responses to targets of zero contrast) are just as likely to be caused by stimulation of mechanisms that are not. Both the total number and the distribution1 of responses can affect the clarity of an estimated template, but any difference between the shapes of templates estimated from trials with zero-contrast targets and trials with nonzero-contrast targets may be due to uncertainty regarding the mechanisms most sensitive to the target and/or an inability to ignore mechanisms insensitive to the target. 

Psychophysical Experiments

General Methods

The Psychophysica (Watson & Solomon, 1997) software used in these experiments is available at http://vision.arc.nasa.gov/mathematica/psychophysica.html. Stimuli were displayed on an Apple Multiple Scan 1705 monitor using only the green gun. A video signal with 12 bit precision was attained using an ISR Video Attenuator, which conforms to the specifications described by Pelli and Zhang (1991). Display resolution was 22.6 pixels/cm. A frame rate of 120 Hz allowed target and mask to be presented on alternate frames with no visible flicker. In some experiments, images were magnified by pixel replication to reduce limitations imposed by monitor bandwidth. All of the experiments were performed between late 1998 and early 2001. During that time, the maximum luminance of the monitor l_max, decreased from 40 to 26 cd m⁻². Every few months the monitor was re-calibrated and the background luminance was set to one half of the maximum luminance. Minimum luminance l_min, was always < 0.1 cd m⁻². 

In the discussion below, I use the conventional decibel scale of stimulus contrast: if m is the maximum available contrast, then an x dB stimulus is one that has a contrast of m 10^x/20. I also frequently cite the correlation r_x,y between two vectors x and y (usually representing a target and an estimated template):  

 

(10)

A correlation of 1 indicates a perfect match; −1 indicates that the two vectors represent photographic negatives of each other. 

Experiment 1

Background

Like the neurones in V1, detection mechanisms are thought to have little sensitivity outside a small region of visual space and a narrow band (1–2 octaves) of spatial frequency (De Valois & De Valois, 1988; Graham, 1989). Previous research (Abbey et al., 1999) sought evidence for use of band-limited templates when detecting a broadband stimulus. Experiment 1 replicates the conditions of the previous study and employs MLA to evaluate differences between the templates used by various observers including the ideal observer. 

Method

In this experiment, images were magnified by 5 in each dimension (see “General Methods”). The viewing distance was such that each magnified pixel subtended 37 s of visual angle. Each trial contained two 0.2 s displays separated by a 0.5 s interval. The two displays in each trial of this experiment contained different (32 × 32 pixel) samples of visual noise. The luminance of each pixel in each sample was independently drawn from a Gaussian distribution, with a standard deviation equal to one quarter of the available range of luminances. The mean of this distribution was the background luminance. The target was a Gaussian blob (σ = 0.15 degrees). This target was added, pixel-by-pixel, to one of the displays in each trial. An adaptive staircase (Watson & Pelli, 1983) determined the target contrast required for observers to identify the target display with 75% accuracy. The accuracy of each response was indicated with a tone. Viewing was binocular and there were four observers: J.A.S. (the author), A.J. and S.Y.A. (experienced psychophysical observers, naïve to the purpose of this experiment) and N.E., who had no previous experience with psychophysical experiments. 

Results and Discussion

Each observer completed 2,000 trials. The first 200 were discarded and templates were estimated (using the standard technique) from the remainder. S.Y.A. was the most sensitive observer. She required a target contrast of −23 dB to attain 75% accuracy. A.J., J.A.S., and N.E. required −22 dB, −20 dB and −18 dB, respectively. 

The green curves in Figure 1a–1d show how template intensities vary with distance from the center of the display. (They have been scaled to have unit height.) For reference, the red curves show how the target’s intensity varies with distance from its center. (They have been scaled to have minimal distance from the green curves.) Consistent with the previous report (Abbey et al., 1999), some templates are noticeably narrower than the target. To determine whether these templates are significantly narrower than the target, two template models were compared. In the more general model, the template was allowed to be any bright Gaussian blob. In the less general model, the template was forced to have the same space constant as the Gaussian target. MLA revealed that the responses from three of the four observers (A.J. excluded) were significantly more consistent with Gaussian templates having smaller space constants than that of the target (p < 3 × 10⁻⁸). 

Figure 1

 

2AFC blob-detection. The red curves describe the target, the green curves describe templates as estimated by standard analysis, and the blue curves describe the best-fitting Gaussian templates whose background is arbitrarily dark, as determined by MLA. MLA reveals significant differences between the templates used by different observers.

The discrete (2-D) Fourier transform was used to reveal the spectral content of the estimated templates. The real parts of the amplitude spectra are shown in Figure 1e–1h. Consistent with the previous report (Abbey et al., 1999), low-frequency suppression appears to be present in some of the templates. To determine whether this suppression was significant, another comparison was made. In this comparison, the less-general model was identical to the more general model in the previous test: the template was allowed to be any bright Gaussian blob. The more general model allowed templates composed of bright Gaussian blobs on arbitrarily dark backgrounds. This more general template is like a difference of Gaussians, where the negative Gaussian has infinite extent. MLA revealed that the more general model provided a significantly better account of the responses from two of the four observers: J.A.S. and S.Y.A. (p < 0.03). (The other observers’ responses would have been better fit by the general model had it allowed bright backgrounds.) Templates satisfying the constraints of the more general model, which maximize the joint likelihood of each observer’s responses, are shown Figure 1 (blue curves). 

Because the templates used by J.A.S. and S.Y.A. exhibit significant low-frequency suppression, whereas A.J.’s template is not significantly different from the target, it seems likely that there are significant individual differences between the templates used by different observers. To test for significant individual differences, each observer’s responses were fit with three additional models, each constraining the template to be identical to the (blob minus background) template that maximized the likelihood of another of the four observers’ responses. These fits were then compared with that of the maximally likely blob-minus-background template. Twelve comparisons thus test the 12 null hypotheses that the parameter values describing each observer’s template are the same as those derived from each of the other observers’ responses. Approximate p values for each of these 12 tests are given in Table 1. As a guide to reading Table 1, consider the entry in the second column of the first row. This entry denotes that the parameter values describing S.Y.A.’s template are significantly different from those derived from A.J.’s responses (p < 10⁻¹⁶). Because the parameter values describing A.J.’s template are significantly different from those derived from the other three observers’ responses and the parameter values describing each of their templates are significantly different from those derived from A.J.’s responses, it is reasonable to conclude that A.J.’s template is significantly different from those used by the other three observers. On the other hand, because the parameter values describing S.Y.A.’s template are not significantly different from those derived from J.A.S.’s responses (p ≈ 0.12) nor are the parameter values describing J.A.S.’s template significantly different from those derived from S.Y.A.’s template (p ≈ 0.23), it is reasonable to conclude that S.Y.A. and J.A.S. use templates that are not significantly different.  

Table 1

 

p values testing the hypotheses that the parameter values describing each observer’s template are identical to those derived from each of the other observers’ responses.

	Responses
Templates	S.Y.A.	A.J.	J.A.S.	N.E.
S.Y.A.		<10−16	0.12	3×10−4
A.J.	3 × 10−13		10−13	10−10
J.A.S.	0.23	<10−16		0.09
N.E.	0.06	3 × 10−9	0.4

Experiment 2

Background

Gabor patterns have been used to describe the receptive fields of simple cells for decades and are therefore good candidates for the preferential stimulation of individual detection mechanisms. If a target were to succeed in stimulating a single detection mechanism and that mechanism’s template were sufficiently different from the ideal, then standard analysis should produce an image that is significantly different from the target. 

I previously reported a mismatch between a circular horizontal Gabor target and templates estimated using the standard analytical technique (Solomon & Morgan, 1999). Consistent with other recent psychophysics (Polat & Tyler, 1999), I found that the templates were elongated horizontally. However, subsequent MLA revealed that this elongation was significant for only two of four observers. Nonetheless, I will describe the experiment in detail here (as Experiment 3) because its results conclusively reject the similarity model of pattern masking (described above). 

Ahumada and Beard (1999) also estimated visual templates for detecting a circular horizontal (2 cycle/degree) Gabor. They used a yes/no procedure and found that neither the template estimated from the target-present trials nor the template estimated from the target-absent trials appeared to be systematically different to the target. However, when they repeated their experiment using a 16 cycle/degree Gabor, the template estimated from the target-absent trials appeared to be totally uncorrelated with the target (r ≈ 0). In Experiment 2, I replicate this result using a 3 cycle/degree Gabor presented at an eccentricity of 3 degrees. Ahumada and Beard concluded that observers must harbor some uncertainty as to which detection mechanism is most sensitive to high-frequency targets. My results demonstrate that observers are unable to ignore the activity of mechanisms having templates mismatched to the phase of peripheral detection targets. 

Method

In this experiment, images were magnified by 3 in each dimension (see “General Methods”). The viewing distance was such that each magnified pixel subtended 35 s of visual angle. A cuing procedure (see Figure 2) was used to ensure a parafoveal presentation. Two different masks appeared on each trial, one on either side of fixation; 0.18 s before they appeared, the fixation spot was replaced by an arrow that indicated the mask to which the target would be added, if it were to appear. Two identical high-contrast circles that remained visible throughout the experiment further indicated the positions of the masks. When the arrow appeared, the circle to which it pointed reversed contrast. Masks were 20 × 20 pixel samples of visual noise. Each pixel of noise was independently drawn from a uniform distribution over half the available range of luminances. The target was a Gabor pattern (see Figure 3a), the product of a 2.9 cycle/degree sine grating and a Gaussian blob (σ = 0.26 degrees) centered on a bright stripe. Its contrast was 0.63 times that required for a hit rate of 82%, as determined by the adaptive staircase. All other methods were identical to those of Experiment 1. 

 

 

Figure 2

 

Procedure for experiments with parafoveal stimuli. Target and masks were presented at 3 degrees eccentricity. [Movie]

Figure 3

 

Standard analysis of the parafoveal detection experiment for two observers. Target-present trials produced templates (b) that were similar to the target (a, 3 cycle/degree Gabor); no pattern emerged from standard analyses of target-absent trials (c).

Results and Discussion

Observer J.A.S. performed a total of 2,000 trials. Of the 1,000 target-present trials, 866 were conducted with a –24 dB target. (The adaptive staircase placed fewer than 100 target-present trials at any other contrast.) The “yes” frequency for these trials was 0.54. Using the standard analytical technique, a template was estimated from these responses. (It is the sum of noise samples eliciting “yes” responses minus the sum of noise samples eliciting “no” responses.) It is shown in Figure 3b. Its correlation with the target is 0.51. 

Observer M.J.M. performed a total of 2,800 trials. Of the 1,400 target-present trials, 580 were conducted with a −22 dB target. The “yes” frequency for these trials was 0.55. The template estimated from these responses (using the standard technique) is also shown in Figure 3b. Its correlation with the target is 0.34. 

Of M.J.M.’s 1,400 target-present trials, 652 were conducted with a −24 dB target. The “yes” frequency for these trials was 0.51. The template estimated from these responses is similar to that shown in Figure 3b. Its correlation with the target is 0.28. 

The “yes” frequency for the target-absent trials was 0.34 for both observers (J.A.S., 1,000 trials; M.J.M., 1,400 trials). Templates estimated from these responses (using the standard technique) are shown in Figure 3c. Their correlations with the target are 0.02 (J.A.S.) and 0.04 (M.J.M.). Note that each pixel of these estimated templates reflects the sum of 500 (for J.A.S.; 700 for M.J.M.) independent, uniformly distributed random events. Thus, it too can be considered a random event, for which the underlying distribution is Gaussian, with a variance equal to the uniform interval times 500/12 (for J.A.S.; 700/12 for M.J.M.). Compared with this variance, neither estimated template has any pixels with intensities significantly different from zero (p > 0.16, J.A.S.; p > 0.74, M.J.M.). Thus, although standard analyses of target-present trials produced templates that were similar to the target, no pattern emerged from standard analyses of target-absent trials. 

The high correlation between the target and each target-present template implies that target-present responses were influenced by the spatial phase of the noise at the target’s central frequency. However, without further analysis, it is impossible to determine whether noise power at this or any other frequency had any influence on the observers’ responses. To reveal the effects of noise power, the power spectra of those samples eliciting “yes” responses were summed and the power spectra of the other samples were subtracted off. When analyzed in this way, both target-present and target-absent trials produce “spectral templates” that bear a striking resemblance to the power spectrum of the target (Figure 4). Thus, regardless of the target’s actual presence, observers were more likely to say “Yes, I see it” when noise power was concentrated at frequencies close to the target’s central frequency, and they were more likely to say “No, I don’t see it” when noise power was concentrated elsewhere. 

Figure 4

 

Spectral analysis of the parafoveal detection experiment. a. The power spectrum of the target. b. The sum of the power spectra of the samples eliciting hits minus the sum of the power spectra of the samples eliciting misses. c. The sum of the power spectra of the samples eliciting false alarms minus the sum of the power spectra of the samples eliciting correct rejections. All spectral templates resemble the power spectrum of the target.

To understand how a pattern emerged from the spectral analysis of the target-absent trials when no pattern emerged from their standard analysis, consider two noisy patterns that are photographic negatives of each other. If each pattern elicited a false alarm, they would produce a blank image when summed in a standard analysis. On the other hand, because power spectra contain no information about spatial phase, the two patterns have identical power spectra and their sum would look just like either spectrum on its own. The two analyses together indicate that false alarms were elicited whenever the noise contained the appropriate horizontal frequencies, regardless of their spatial phase. 

Responses that depend on the frequency content of the stimulus but not its phase content are inconsistent with any response rule, such as Equation 2, in which the observer looks for a sufficient match between the stimulus and just one template. An alternative rule that can explain the results of Experiment 2 is one in which the observer looks for a sufficient match between the stimulus and each of a set of templates. If any of them exceed some criterion, then the observer responds with a “yes.” When the target is absent, the match between the stimulus and each template may equally often exceed the criterion In that case, the standard analysis will produce an image that resembles the sum of all these templates. When the target is present, the match between the stimulus and one particular template (the best-matched template) will exceed the criterion more often than the match between the stimulus and any of the other templates. In this case, the standard analysis will produce an image that is dominated by the best-matched template. 

If, as suggested above, (psychophysical) templates correspond to (physiological) receptive fields, then our observers’ inability to monitor the output of a single phase-sensitive mechanism corresponds to a lack of direct conscious access to the activities of individual simple cells, which are, by definition, phase-sensitive (Hubel & Wiesel, 1959). Alternatively, it may be that phase-insensitive cells somehow have such a high signal-to-noise ratio that the output of simple cells becomes irrelevant. 

Experiment 3

Background

See Experiment 2, “Background.” 

Method

This is the only experiment in which viewing was monocular rather than binocular. The two displays in each trial contained the same 36 × 36 pixel sample of visual noise, centered on fixation. The luminance of each pixel in each sample was independently drawn from a uniform distribution over one eighth of the available range of luminances. (One observer also tried a larger interval: one half of the available range of luminances.) The Gabor target (see Figure 5) was the product of a 4.9 cycle/degree sine grating and a Gaussian blob (σ = 0.14 degrees) centered on a bright stripe. This target was added, pixel-by-pixel, to one of the displays in each trial. The adaptive staircase determined the target contrast required for observers to identify the target display with 75% accuracy. All other methods were identical to those of Experiment 2. 

Figure 5

 

Target t (a) and sum of error-producing samples e for observer S.C.D.(b), for a simulation of the sensory-threshold model (c), and for a simulation of the similarity model (d) in a twinned-noise experiment. The negative correlation between t and e is consistent with the sensory-threshold theory of pattern masking. It is inconsistent with the similarity theory of pattern masking.

Results and Discussion

For each observer, the target t was negatively correlated with sum of the samples present on trials that produced incorrect responses e. Figure 5b shows one example. For J.A.S., r_t,e = −0.16 (205 incorrect responses with a −20 dB target); for S.C.D., r_t,e = −0.23 (276 incorrect responses with a −24 dB target); and for A.C.M., r_t,e = −0.22 (308 incorrect responses with a −20 dB target). I.M.E. was the inexperienced observer, and 3,000 responses were collected from her. The first 1,000 yielded r_t,e = −0.11 (286 incorrect responses with a −20 dB target); the second 1,000 yielded r_t,e = −0.17 (217 incorrect responses with a −20 dB target); and the final 1,000 yielded r_t,e = −0.21 (290 incorrect responses with a −22 dB target). For J.A.S., with the high-contrast noise, r_t,e = −0.14 (240 incorrect responses with a −10 dB target). 

Predictions of the sensory-threshold and similarity theories of pattern masking (see above) are also illustrated in Figure 5. For each of the theories, the performance of a model observer was simulated. These observers were given the same samples of noise as S.C.D. Each observer’s template was identical to the target. Similarity Theory’s response rule is described in Equation 9. The model observer had no internal noise. Sensory-threshold theory is described in Equations 6 and 8. The parameter t was set to 0, and the variance of the observer’s internal noise was assumed to be sufficiently low for a correct response from any trial i in which t^t(t+n_i) > 0. Therefore, on trials when t^t(t+n_i) < 0, each response was effectively selected from a Bernoulli process with a 50% success rate. 

The performance of the sensory-threshold model is qualitatively similar to the performance of each human observer’s: samples eliciting incorrect responses form an image that is negatively correlated with the target. On the other hand, samples eliciting incorrect responses from the similarity model form an image that is positively correlated to the target. On the basis of these results, the similarity theory of pattern masking can be rejected. 

Adopting the response rule described in Equations 6 and 8, I calculated the maximum likelihoods for all responses from each observer given two templates: the ideal template (i.e., the target) w_i and another Gabor template w_h, with freely varying vertical and horizontal spreads σ_y and σ_x. Although a good case could be made for a horizontally elongated w_h from either the results of observer J.A.S.2, neither A.C.M.’s nor I.M.E.’s responses were significantly more likely with w_h ≠ w_i. S.C.D.’s responses were significantly more likely with w_h ≠ w_i, however, the aspect ratio (σ_x/σ_y) of his maximally likely w_h was a mere 1.3; not exactly overwhelming evidence for a horizontally elongated template. (Note that the statistics described above are not inconsistent with the hypothesis that all four observers used the same template. An analysis of Experiment 3 analogous to that shown in Table 1 has not yet been performed.) 

Experiment 4

Background

Psychophysical reverse correlation can be applied to virtually any task that can be made more difficult by the addition of visual noise. Orientation discrimination has long been thought to involve an opponent process that compares the activities in detection mechanisms having differently oriented templates (Regan & Beverley, 1985). In Experiment 4, I use reverse correlation to reveal exactly which detection mechanisms are used by the opponent process. Moreover, Experiment 4 provides evidence that these detection mechanisms are subject to a simple, hard threshold as described in Equation 8. 

Method

Methods were identical to those of Experiment 2 with the following exceptions. Two Gabor patterns appeared on every trial (one in the left position, one in the right position; see Figures 2 and 6a). Each was the product of a 2.5 cycle/degree sine grating and a Gaussian blob (σ = 0.26 degrees) centered on a bright stripe. One Gabor pattern, the distracter, was horizontal. The other, the target, was tilted 11 degrees clockwise or counterclockwise from horizontal. The observer had to decide whether the target’s tilt was clockwise or counterclockwise. Figures 2 and 6 show the cued condition in which a unidirectional arrow indicated which of the two Gabor patterns was the target. In the uncued condition, this unidirectional arrow was replaced by an uninformative bidirectional arrow (see Figure 7a). Adaptive staircases converged on the contrast (applied to both Gabor patterns) required for 75% accuracy in each condition. J.A.S. and A.C.M. (a naïve but experienced psychophysical observer) performed between 1,000 and 1,100 trials in each condition. 

Figure 6

 

Stimulus, results, and simulation for an orientation-discrimination experiment with a spatially cued target. a. A counterclockwise target and a horizontal distracter, but not the random noise masks that accompanied them. b. Their power spectra. c, e, and g. Sums of error-producing samples in target (left) and distracter (right) positions for J.A.S., A.C.M., and a model observer, respectively. Samples added to clockwise targets were flipped prior to summation. d, f, and h. The power spectra of the sums in c, e and g, respectively. These images provide evidence that those samples that reduced the intensity of the true (counterclockwise) target also impaired performance, but these images do not indicate that those samples resembling the false (clockwise) target had any effect on performance at all.

Figure 7

 

Stimulus, results, and simulation for an orientation-discrimination experiment with an uncued target. a. A counterclockwise target and a horizontal distracter, sans random noise masks. b. Their power spectra. c, e, and g. The sums of error-producing samples in target (left) and distracter (right) positions for J.A.S., A.C.M., and a model observer, respectively. Samples added to clockwise targets were flipped prior to summation. d, f, and h. The power spectra of the sums in c, e, and g. They are more oblique than those of the two targets. Samples that masked the true target impaired performance only in the target position. Samples that resembled the false target impaired performance only in the distracter position. These results imply a hard threshold for detection. The model uses a hard threshold and an optimized opponent mechanism (see text).

Results and Discussion

Figures 6c and 6e show the sums of the noise samples present on cued trials when observers J.A.S. and A.C.M. responded incorrectly. The left sides of Figures 6c and 6e show the sums of samples from the target position; the right sides show the sums of samples from the distracter position. If the target were tilted clockwise from horizontal (instead of counterclockwise as in Figure 6a), the sample was flipped (so that the top row of pixels became the bottom row of pixels, etc.) prior to summation. Figures 6d and 6f show the power spectra of the sums in Figures 6c and 6e. When these spectra are compared with those of the target and distracter (see Figure 6b), it becomes apparent that the observers responded incorrectly when the target’s mask contained frequencies similar to but more oblique than those in the target itself. As is apparent from Figures 6c and 6e, these frequencies impaired performance when their phases were opposite to those in the target (i.e., they form a Gabor pattern with a central black stripe). 

A similar analysis of the uncued trials (Figure 7) reveals a similar though somewhat less distinct pattern of results. As is apparent from Figures 7d and 7f, both observers also responded incorrectly when the distracter’s mask contained frequencies similar to but more oblique than those in the false target. (Note: if the target’s tilt were counterclockwise, then the false target’s tilt would be clockwise, and vice versa.) As is apparent from Figures 7c and 7e, these frequencies impaired performance when their phases were similar to those in the false target (i.e., they form a Gabor pattern with a central white stripe). Simply, two types of noise impaired performance: noise that masked the true target and noise that resembled the false target. The former was effective only in the target position and the latter was effective only in the distracter position. 

Assume that on cued trials an observer used two detection mechanisms, one preferring the true target, the other preferring the false target. When the output of the former exceeded the output of the latter, the observer responded correctly. If output had been proportional to input, then those noise samples that changed the latter mechanism’s output should have affected performance just as effectively as those that changed the former mechanism’s output. They did not; thus output must have increased faster with near-threshold inputs than it did with sub-threshold inputs. (This argument is formalized in “2.”) 

Further, assume that on uncued trials, the observers chose whichever mechanism produced the greatest output in either location. If output had continually accelerated with all small inputs (e.g., output = input^p, p > 1), then the only noise samples that should have affected performance are those that changed the output of the most strongly stimulated mechanism: the one stimulated by the true target. Because samples that changed the output of the mechanism preferring the false target (in the distracter location) also affected performance, there must have been a range of near-threshold inputs with which output increased linearly. (This argument is formalized in “3.”) 

Thus, the results from cued trials and the results from uncued trials each constrain the relationship between input and output in a different way. The simple, hard threshold, as described in Equation 8, satisfies these constraints. 

Of the detection mechanisms whose preferred stimuli are more (or less) oblique versions of the two targets, the two whose outputs differ the most to a genuine target are those whose preferred stimuli are tilted ±28 degrees from horizontal. The average (unsigned) tilt of the three Gabor patterns (whose spatial layout was identical to that of the target except for orientation and possibly contrast polarity) that best fit the three sums in Figures 6b and 7b (as determined and weighted by correlation; the right side of Figure 6b was excluded) is also 28 degrees. The corresponding average derived from A.C.M.’s results is 26 degrees. Thus, observers seem to use the best two mechanisms. 

For each condition of Experiment 4, 2,000 trials were simulated. As described above, the model observer used two noise-free detection mechanisms, their templates identical to the two targets except that their orientations were ±28 degrees from horizontal. Its response was determined by whichever mechanism produced the greatest output (in either location, for uncued trials). Each detection mechanism employed a hard threshold, as described in Equation 8. The maximum gain of each template was arbitrarily set to 1. A value of 0.25 for the parameter t (see Equation 8) produced panels g and h in Figures 6 and 7. These images are qualitatively similar to those produced by the human observers. 

In Experiment 2, it was shown that noise power could affect responses even when its phase was irrelevant. This was ascribed to observers’ intrinsic uncertainty regarding phase. To determine if phase uncertainty played a role in Experiment 4, the average power spectra of those samples eliciting incorrect responses were calculated. (Nota bene: this differs from the power spectra of the summed samples shown in Figures 6 and 7; see Experiment 2 for details of this spectral analysis.) These average power spectra are shown in Figure 8. They provide scant evidence for an effect of noise power at any frequency. Thus, for example, in the target location, observers may have monitored mechanisms preferring the false target’s orientation and frequency (and a variety of spatial phases); however, none of these mechanisms were stimulated with sufficient frequency to make an impact on the results of this experiment. 

Figure 8

 

Spectral analysis of Experiment 4. a. The power spectra of the counterclockwise target (left) and distracter (right). b – e. The sums of the power spectra of each error-producing sample in the corresponding location. (Samples added to clockwise targets were flipped prior to summation.) b: J.A.S. cued condition; c: A.C.M. cued condition; d: J.A.S. uncued condition; e: A.C.M. uncued condition. No clear pattern emerges from this analysis.

Conclusion

Reverse correlation is a versatile tool for psychophysics. Even highly nonlinear detection mechanisms, such as that subserving parafoveal wavelet detection, can be described when the random masks are appropriately transformed in the analysis. Even more conventional detection mechanisms, such as those subserving the detection of noisy low-frequency patterns in central vision, display nonlinear properties (e.g., sensory thresholds) when individual trials are analyzed. Finally, the detection mechanisms that putatively form the first stage of parafoveal orientation discrimination apparently share the sensory threshold exhibited by those responsible for foveal patterns. 

Acknowledgments

This review is the fruit of many discussions with various colleagues. Al Ahumada, Art Burgess, Miguel P. Eckstein, Michael J. Morgan, Ariella Popple and especially Craig Abbey provided incisive comments on my work, some of which was presented at The Association for Research in Vision and Ophthalmology’s annual meeting in Ft. Lauderdale, Florida (both 1999 and 2000). In addition, I would like to thank S.Y.A., S.C.D., I.M.E., N.E., A.J., A.C.M., and M.J.M. for their observations. Commercial Relationships: None. 

Footnotes

Footnotes

¹ The closer the frequency of “yes” responses is to 0.5, the clearer the estimated template will be.

Footnotes

² With low-contrast noise, his responses were significantly (p < 0.009) more likely with w_h ≠ w_i and the aspect ratio of the maximally likely w_h was 2.2; with high-contrast noise, his responses were significantly (p < 0.0008) more likely with w_h ≠ w_i and the aspect ratio of the maximally likely w_h was 2.3.

Appendix 1

Using logic similar to that displayed in Abbey et al. (1999), here I prove that Equation 2 implies that the standard analysis of either the target-present or the target-absent trials from a Y/N detection experiment produces an unbiased estimate of the template. It will suffice to show that the expected contribution of either type of trial toward this estimate is itself a scaled version of the template. Let n_i represent the mask present on trial i. n_i is added to the template estimate if i elicits a “yes” response; otherwise, n_i is subtracted from the template estimate. The expected contribution of trial i toward a standard analysis is therefore <n_i sgn[w^ts_i+η−c]>_η, where w represents the template, c represents the internal criterion, η represents a sample of the internal noise and s_i represents the stimulus. Because for target-absent trials, s_i=n_i, and for target-present trials, s_i=n_i+t, the expected contribution of any trial toward a standard analysis can be rewritten as <nsgn[w^tn+η−ĉ]>_n,η, where ĉ is a constant that is different for target-present and target absent trials. (Note that the target must have the same contrast for all trials used in the standard analysis). Thus, it remains for me to prove  

 

(A1)

Where k is a constant. 

The first assumption I need to make explicit is that the internal noise is independent of the mask. This allows a simplification of the left-hand side of 1.  

 

(A2)

 

Next, the noise is rotated: let n = W^t^n, where Ww = ‖w‖e₁, with e₁ defined as the vector that is 1 in its first element and 0 in all others. Note that if all the elements of n have zero mean, then so will all the elements of ^n. Using ^n, 2 can be re-written as  

 

(A3)

where ^n₁ is the first element of ^n. 3 can be expanded if we define e_j as the vector that is 1 in its jth element and 0 in all others:  

 

(A4)

 

This last step is because all the elements of ^n have zero mean. If η were uni-variate normal and n were multi-variate normal, then a precise scalar value for the expectation in 4 could be determined. However, it is not necessary to determine the precise value. As long as all the elements of n have zero mean and are independent of η, the expectation will be some constant ^k. Thus  

 

(A5)

Where k ≠ ^k. Q.E.D. 

Appendix 2

Let a and c represent the counterclockwise target and the clockwise targets in Experiment 4, respectively. Assume that in the cued condition the observer responds “counterclockwise” when the output of a detection mechanism sensitive to a exceeds the output of another sensitive to c. That is, the observer responds “counterclockwise” when  

 

(A6)

where s represents the stimulus in the cued location, f is some non-decreasing function, and η represents a sample of the internal noise. The templates w_a And w_c, are constructed such that  

 

(A7)

 

Now consider what happens when the target is clockwise, i.e. s = c + n. If f were linear, then incorrect responses would occur whenever w_a^tn − w_c^tn > η′, for some random η′. (NB: the PDFs of the random variables giving rise to η′ and η′ would have the same shape.) However, the results of Experiment 4 indicate that incorrect responses occur when c^tn (and thus w_c^tn) is negative; w_a^tn does not seem to matter. Thus f cannot be linear. 

In particular, let p be some small positive number, 0<p<<1. The results of Experiment 4 indicate that s=c−pw_c would cause more errors than s=c+pw_a. Thus,  

 

(A8)

Using the identities in A7, 8 can be rewritten as  

 

(A9)

and rearranged as  

 

(A10)

But,  

 

, so 10 implies  

 

(A11)

where a < b < c. That is, f(x) increases faster than x. 

Appendix 3

Consider what happens when the target is clockwise in Experiment 4’s uncued condition. Let c represent that clockwise target and let d represent the distracter. Thus the stimulus in the target position can be described c+n₁ and the stimulus in the distracter position can be described d+n₂. Finally, assume that the observer responds incorrectly when  

 

(A12)

where η represents a sample of the internal noise and the templates w_a and w_c, conform to 7. To determine whether f(x) = Max{x,c} or f(x) = Sgn(x)|x|^p is a better representation, assume for the time being that  

 

(A13)

Given the difference between c and d, w_c^t(c + n₁) > w_c^t (d + n₂) and w_a^t(d + n₁) > w_a^t (c + n₂) for virtually any noise samples n₁ and n₂. Because f is non-decreasing, we can simplify 12 to say that the observer will respond incorrectly on clockwise trials when  

 

(A14)

These last two equations can be combined to yield  

 

(A15)

If p were much greater than 1, we would not expect to find any effect of n₂ on response accuracy. However, the results of Experiment 4 indicate that incorrect responses occur not only when c^tn₁ (and thus w_c^tn₁) is negative, but also when a^tn₂ (and thus w_a^tn₂) is positive; thus p must not be much greater than 1. In fact, when p is exactly 1, and w_a and w_c are constrained to be rotated versions of d, simulation indicates that w_c^tc−w_a^tc is maximized when w_a and w_c are rotated ±28 degrees, a value that conforms to the result of Experiment 4 (see main text).