Odd-men-out are poorly localized in brief exposures

Abstract

Signal detection theory (SDT) asserts that sensory analysis is limited only by noise, and not by the number of stimuli analysed. To test this claim, we measured the accuracy of visual search for a single tilted element (the target) among 7 horizontal elements (distractors) using several different exposure durations, each terminated by a random noise mask. In the uncued condition, each element was a potential target. In the cued condition only 2 were. SDT predicts that location errors should be evenly distributed among all distractors. For long exposures (eg, 5.0 seconds), this prediction was confirmed, and SDT could simultaneously fit uncued and cued accuracies. For short exposures (eg, 0.1 seconds), errors were concentrated among distractors adjacent to the target, and, unless modified to account for this, SDT underestimated the difference between uncued and cued accuracies. Therefore, when the time available for search is brief, odd-men-out (ie, featural discontinuities) can be seen, but their positions can be only roughly estimated.

Introduction

To find a tilted target among horizontal distractors, the visual system must estimate the orientation of each element. We wanted to know how the accuracy of these estimates depends upon their number and exposure duration. Using the method employed by Shaw (1980), on every trial, 8 Gabor patterns were displayed. In one half of the trials (the uncued trials), any pattern could be the target. In the other half (the cued trials), only 2 patterns (on opposite sides of fixation) could be the target (Figure 1). 

 

 

Figure 1

 

QuickTime movie of trial sequence. Eight Gabor patterns appear for 0.1, 0.2, 0.5, 1.0, or 5.0 seconds. Eight or (as in this case) 2 limbs of a spatial cue precede the Gabor array by 1 second. A random noise postmask follows it immediately. Observers must report the position of the target and its direction of tilt (clockwise or counterclockwise).

For each duration, each observer’s performance was fit with signal detection theory (SDT). SDT provides a useful benchmark because it asserts that the visual system can estimate with the same accuracy any number of orientations in parallel. Specifically, SDT asserts that Gaussian noise perturbs the perceived orientation of each element. The target can be located by selecting the element having the greatest apparent tilt (either clockwise or counterclockwise). When the variance of the noise is specified, psychometric functions (frequency of correct location versus tilt) for both uncued and cued conditions can be calculated. 

Methods

Stimuli were generated by a http://www.crsltd.com/ VSG graphics card (Kent, UK) with 14-bit luminance resolution and displayed on a gamma-corrected http://www.mitsubishi.com/ (London, UK) DiamondPro display (resolution: 512 × 512; display area: 23.6 × 23.6 cm; frame rate: 100 Hz; mean luminance: 15 cd/m²; viewing distance: 73 cm). Both observers had normal vision. One observer (A.J.) was naïve to the experiment’s purpose, and the other observer (M.J.M.) is an author. 

Each element was an odd-symmetric Gabor pattern: a 3.8 cycle/degree sinusoidal grating multiplied by a circular Gaussian with a space constant of 0.175 degree and an amplitude 90% of the display’s dynamic range. The elements were equally spaced at 45-degree intervals around an isoeccentric circle with a radius of 3.5 degrees, and, thus, had a centre-centre separation of 2.67 degrees of visual angle. Each patch was also surrounded by a square box (each side was 1.05 degrees) to aid spatial localization (Figure 1). To limit the time available for visual processing, each box was filled with maximum-contrast 2-D white noise immediately upon element offset. 

Two trial sequences are illustrated in Figure 1. After each display, the observer entered a 2-digit number on the numeric keypad of the computer. The first digit indicated the orientation of the target (1 for clockwise and 2 for counterclockwise). The second digit (1–8) indicated the target’s position in an analogue manner. As soon as the observer entered the second digit, the central fixation point (a white asterisk) disappeared and was replaced by the cueing array. Observers were instructed not to move their eyes during the display. 

Trials were blocked by exposure duration. For each duration, the 2 conditions (uncued and cued) were run simultaneously, with trials randomly interleaved. Five levels of target tilt were also interleaved. Trials were run in blocks of 200, with rests between blocks for the observer. Testing continued until 100 trials had been collected in each condition with each level of tilt. 

Results

Data (points) and fits (solid curves) for 0.1- and 5.0-second exposures are shown in Figure 2. Although the fits are good for 5.0-second exposures, SDT cannot account for the difference between uncued and cued accuracies with 0.1-second exposures. For large target tilts, SDT underestimates the difference between cued and uncued accuracies. 

Figure 2

 

Psychometric functions for location. Left panels are from observer M.J.M., and right panels are from A.J. Top panels are from 0.1-second exposures; bottom panels are from 5.0-second exposures. Results from both uncued trials (lower points) and cued trials (upper points) appear in each panel. Error bars contain the 95% confidence intervals. Solid curves show unmodified SDT, when best fitting the uncued and cued results simultaneously. Dotted curves show the fit of SDT when modified to produce the proximity effect. Data from Figure 4 were included in these latter fits. Unmodified SDT works well for 5.0-second exposures, but some modification is required for it to fit the results from 0.1-second exposures.

If location errors were distributed evenly among distractors, then, in the uncued condition, only 2 of 7 (29%) would fall next to the target. Yet we found that more than 40% of location errors fell next to the target when the shortest exposures were used. Figure 3 shows that this proximity effect declines with long exposures. Figure 4 shows how the frequency of these “adjacent errors” depends upon target tilt for 0.1- and 5.0-second exposures. 

Figure 3

 

The proximity effect. For each observer (M.J.M.: boxes; A.J.: stars) and duration, we plotted the frequency of mislocations directed to elements adjacent to the target.

Figure 4

 

Fitting the proximity effect. As in Figure 2, left panels are from observer M.J.M. and right panels are from A.J. Top panels are from 0.1-second exposures; bottom panels are from 5.0-second exposures. Error bars contain the 95% confidence intervals. Unlike Figure 2, the frequencies of adjacent errors (on only uncued trials) are plotted here. Solid curves show unmodified SDT when best fitting these data and those of Figure 2. Dotted curves show the fit of SDT when modified to produce the proximity effect. Unmodified SDT works well for 5.0-second exposures, but some modification is required for it to fit the results from 0.1-second exposures.

The fit of SDT to the short-exposure data can be improved by allowing a small proportion of the observers responses to be directed to positions adjacent to that of the apparent target. Because the cued elements were on opposite sides of fixation, this modification applies only to the uncued condition. This modified SDT (dotted curves) can simultaneously fit the data in Figures 2 and 4 very well. Unmodified SDT produces a satisfactory fit for 5.0-second exposures, but it cannot predict the large number of adjacent errors observed with 0.1-second exposures (as shown by the solid curves in Figure 4). The fit of the unmodified SDT model is significantly (P < .05) worse than the modified version at fitting the data from each condition except A.J., 5.0 seconds (P ∼ .08; Figure 5). 

Figure 5

 

Goodness-of-fit for location. For each observer (M.J.M: red; A.J.: green) and duration, we computed the maximum likelihoods of modified and unmodified SDT producing the observed frequencies of correct responses and adjacent errors (and thus, nonadjacent errors as well). The natural logarithms of the ratio of these likelihoods are plotted here. Such ratios, when doubled, should follow the χ² distribution (Mood, Graybill, & Boes, 1974), with one degree of freedom. The dashed line illustrates the 0.05 level of significance. The improvement obtained by modifying SDT to allow for the proximity effect declines with exposure duration, yet remains significant except for AJ at 5.0 seconds.

Large differences between uncued and cued location accuracies are a signature of limited-capacity models, which posit an inverse relationship between the accuracy and the number of simultaneous feature estimates. For every cued accuracy, there is a corresponding uncued accuracy below which no (otherwise unmodified) SDT model, unconstrained by the assumption of Gaussian noise, can simultaneously predict (Shaw, 1980). This boundary is drawn in Figure 6 along with the 0.1-second data and their fit with the modified SDT model from Figure 2. Both the data points and the curves generated by modified SDT cross the boundary near the upper right corner in each panel. This means that the proximity effect can masquerade as a capacity limitation. Had we not discovered the proximity effect, we might have rejected SDT because of these data. 

Figure 6

 

Uncued versus cued performance. Each panel replots the data and their fit by the modified SDT model (dotted black curves) from the top panels in Figure 2. Once again, the error bars contain the 95% confidence intervals. Also shown are the prediction of unmodified SDT (solid black curves) and the boundary (solid red curves), above which fall all (otherwise unmodified) SDT models unconstrained by the assumption of Gaussian noise. Although the data cross the boundary, so does the curve generated by modified SDT.

If, as we claim, the proximity effect were responsible for unmodified SDT’s underestimate of the difference between uncued and cued accuracies, then, had cued elements been adjacent to each other instead of on opposite sides of fixation, we should have observed less of a difference than we did. By corollary, the target should be easier to find when the cued elements are on opposite sides of fixation than when the cued elements are next to each other. In a subsidiary experiment, we confirmed these predictions using 3 interleaved conditions: (1) cued elements on opposite sides of fixation (as before); (2) cued elements next to each other; and (3) uncued (also as before). The exposure duration was 0.1 second. Results appear in Figure 7. Unmodified SDT (solid curves) makes the same predictions for the first 2 conditions. Although it does a reasonable job of fitting data from the latter 2 conditions, SDT consequently underestimates the accuracy of the first condition. The modified SDT model (dotted curves) can fit all 3 conditions simultaneously. 

Figure 7

 

Accuracy/proximity link. The left panels show location accuracy from trials in which (as in Figure 1) the cued elements are on opposite sides of fixation. The center panels show location accuracy from trials in which the cued elements are next to each other. The right panels show location accuracy and the frequency of adjacent errors (red points) from uncued trials. Error bars contain the 95% confidence intervals. Solid curves show unmodified SDT when best fitting all accuracies simultaneously. Dotted curves show modified SDT when best fitting these same accuracies plus the frequencies of adjacent (and nonadjacent) errors in the uncued condition. Targets were easier to find when the cued elements were on opposite sides of fixation than when they were next to each other.

In addition to locating the target’s position, our observers were asked to identify the target’s orientation (clockwise or counterclockwise) following each display. Using similar methods, Baldassi & Burr (2000) noted that orientation thresholds for identification were sometimes less than those for location. Our results with cued displays (and some of M.J.M.’s results with uncued displays) confirm this finding. (If the 2 types of thresholds were actually identical, the odds of all 10 identification thresholds being less than their corresponding orientation thresholds would be greater than 1000:1.) Figure 8 shows the ratios of these thresholds, calculated from the psychometric data of our main experiment. All data points, except those from A.J. in the uncued condition, fall above the dashed lines at a ratio of 1. 

Figure 8

 

Threshold ratios. Each symbol shows threshold for location divided by threshold for identification. Uncued results were used for the left panel; cued results for the right. Boxes were computed from M.J.M.’s data, and stars were computed from A.J.’s. Boxes were shifted slightly left and stars were shifted slightly right for legibility. Error bars contain the 95% confidence intervals. Missing symbols indicate that the confidence interval could not be sufficiently constrained on the basis of our data. Dashed lines indicate a ratio of 1. The red line indicates the prediction of SDT when observers’ identifications are based on the mean apparent tilt. The green line indicates the prediction of SDT when observers’ identifications (as well as localizations) are based on the greatest apparent tilt. Threshold ratio for SDT’s ideal observer must fall between the green and blue lines. For the cued condition, all of these models are equivalent (solid black line). For each model, the predicted ratio decreases with the number of possible targets. Responses based on the maximum apparent tilt (and thus ideal decisions) always predict ratios greater than 1.

The colored lines in Figure 8 represent threshold ratios from 3 hypothetical observers for whom location and identification are limited by the same noise, that which corrupts each local estimate of orientation. All 3 hypothetical observers adhere to unmodified SDT when locating the target. The red line represents the threshold ratio from an observer who reports the mean apparent orientation. The green line represents the threshold ratio from an observer who reports the maximum apparent orientation. The green line thus also serves as a lower boundary for the ideal observer’s threshold ratio, for which there is no general solution because it depends not only on the variance of the noise but also on its relationship to the range of possible tilts.1 The blue line serves as an upper boundary for the ideal observer. It represents the threshold ratio from an observer for whom just 2 (instead of 10) possible targets appear on every trial, having (identification) threshold tilts in opposite directions. 

For all 3 of these hypothetical observers, the ratio of location threshold to identification threshold increases as the number of possible targets decreases. When, as in the cued condition, there are only 2 possible targets, all 3 of these observers produce the same threshold ratio. It is shown by the solid black line in Figure 8. 

Another way to describe the solid black and green lines is that they represent the threshold ratio from an observer who reports the orientation of the apparent target. Regardless of the number of possible targets, this threshold ratio will always be greater than 1 because the conditional probability of identifying orientation correctly, given that the correct location has been identified, P(I|L) is higher than its inverse P(L|I). Thus, our results cannot be used as evidence for identification without location (Baldassi & Burr, 2000). Most of the symbols in Figure 8 fall below the solid black and green lines. These symbols actually suggest location without identification. Because observers always reported orientation before location, their relatively poor performance in the identification task cannot be attributed to a greater memory load. 

Discussion

There has been some controversy regarding whether or not searches for oriented targets can be aided by attention. Some have concluded that any benefit from a spatial cue can be explained by SDT (Eckstein, 1998; Palmer, 1994; Palmer, Ames, & Lindsey, 1993; Palmer, Verghese, & Pavel, 2000). It should be noted that observers were not required to locate the target (other than to say whether or not it was present in a particular display) in any of these studies. Thus, none of these studies could have revealed a failure of unmodified SDT due to the proximity effect. 

Other studies have found that unmodified SDT does fail to explain some searches for oriented targets (Baldassi & Burr, 2000; Carrasco, Penpeci-Talgar, & Eckstein, 2000; Morgan, Ward, & Castet, 1998; Verghese & Nakayama, 1994). For example, unmodified SDT cannot explain the increase in accuracy that is sometimes found with an increase in the number of distractors (Sagi & Julesz, 1987; Sagi, 1990). Some form of texture processing is thought to mediate such searches (Sagi, 1990; Palmer, Verghese, & Pavel, 2000). 

The proximity effect is also consistent with localization subsequent to any process, such as a textural analysis, in which at least some of the visual systems local estimates of orientation are pooled and thus blurred. Both psychophysical (Watt & Morgan, 1983; Krauskopf & Farell, 1991) and theoretical (Morgan & Aiba, 1985) arguments exist for a deterioration of localization with blur. Another possibility is that our targets were located, not by finding features in a blurred map of orientations, but by finding gradients or borders within it. Fifty percent of the trials in which only 1 of the 2 texture borders (between the target and each adjacent distractor) were found would result in an adjacent mislocation, producing the proximity effect. 

Despite the possibility that our observers used pooled estimates of orientation when deciding the uncued targets location, evidence from previous studies with nearly identical stimuli suggests that the individual estimates were nonetheless available (Morgan, Ward, & Castet, 1998; Baldassi & Burr, 2000). In both of these studies, it was demonstrated that distractors had no effect upon identification thresholds for a single cued target. Thus, while crowding might imply textural analysis (Parkes, Lund, Angelucci, Solomon, & Morgan, 2001), textural analysis need not imply crowding. 

There is, at least, one more potential explanation of the proximity effect. Consider a strategy in which an observer either overtly (using a saccade) or covertly (using attention) focuses upon just one of the potential targets. In trials in which the target is not seen, the opposite element is selected. Although this strategy cannot be completely ruled out, because a formal model would necessarily require at least 1 more free parameter (per duration) than our modified SDT (which provides a good fit), we will not pursue it here.2 Moreover, evidence from a previous study demonstrates that at least 1 of our observers (M.J.M.) is quite capable of inhibiting saccades, even during long displays with drifting stimuli (Morgan, Watt, & McKee, 1983). 

Given that localization is so poor, it may be surprising that identification is (if anything) worse. However, it is entirely possible that the 2 tasks, location and identification, are limited by separate sources of noise within the visual system. Whereas the identification task almost certainly employs filters sensitive to clockwise and counterclockwise orientations, the location selected by an observer may simply correspond to that maximally stimulating a vertical filter. The different varieties of filters may produce different amounts of noise. Finally, location without identification is consistent with the finding that observers can locate discontinuities in seemingly homogenous textures (Kolb & Braun, 1995; Morgan, Mason, & Solomon, 1997). 

Conclusion

The first few hundred milliseconds of vision are seemingly very different from the finished product that appears in consciousness. We can think of the emerging representation of the target as a diffuse “hot spot” in the image, with uncertain location and possibly even less certain feature content. 

Acknowledgments

This work was made possible by a grant from the Engineering and Physical Sciences Research Council (UK) (Grant GR/N03457/01). 

Commercial relationships: N. 

Footnotes

Footnotes

¹ The maximum-apparent-orientation rule is not ideal. If the noise with which the visual system perturbs the true perceived orientation of each element has a high variance, it is nonetheless possible that, on some trials, all of the elements are perceived as being close to horizontal. On such trials, if the majority of those elements had an apparent clockwise tilt, then it would be more likely that the target had a clockwise tilt than a counterclockwise tilt, even if the element having the greatest apparent tilt appeared to be counterclockwise.

Footnotes

² Our modified SDT requires 2 free parameters for each observer and duration: P(B) and σ (see Appendix). In order to model the strategy described here, at least 3 free parameters are required. Each local orientation estimate will now have a different precision. At least 2 parameters (peak and spread) are required to specify the relationship between precision and distance from focus. In addition, a criterion for selecting the opposite element would need to be established.

Appendix

(Model-free) thresholds θ₀ were calculated by (maximum-likelihood) fitting a cumulative Gaussian to the psychometric data:  

 

(1)

where n is the number of possible responses (2 or 8). The parameter σ₀ was allowed to vary freely in each fit. 

In the next paragraph, we specify how to compute the probability of a correct location P(L), and the probability of an adjacent error P(D) given A and B, the independent events that the true target has the greatest apparent tilt and the observer mistakes the source of the maximum sensation adjacent to its true source, respectively. When fitting “modified SDT” to the data, P(B) is allowed to assume any value between 0 and 1. When fitting “unmodified SDT” to the data P(B) is fixed at 0. Below, we specify how to compute P(A) according to SDT. 

Expanding the probability for correct location, we obtain  

 

(2)

 

Because A and B are independent, this becomes  

 

(3)

 

Similarly, the probability of an adjacent error can be written  

 

(4)

 

In the uncued condition, with 8 potential targets, these probabilities become  

 

(5)

and  

 

(6)

 

In the subsidiary experiment, when the 2 potential targets were next to each other, the probability of a correct location is simply  

 

(7)

and P(D) = 1−P(L). When, as in the ‘cued condition’ of the main experiment, the 2 potential targets were on opposite sides of fixation, P(L) = P(A) and P(D) = 0. 

For the target to have the greatest apparent tilt, it must produce a more extreme sensation of tilt than any of the distractors, thus  

 

(8)

where u₁ is an outcome of the random variable U₁, quantifying the apparent tilt of the target (negative values can indicate counterclockwise tilts, whereas positive values can indicate clockwise tilts) and each u_ii ≠ 1 is an outcome of the random variable U_i, quantifying the apparent tilt of a different distractor. Thus, _i where  

 

(9)

where m is the number of possible targets and f_x(u) and F_X(u) denote the probability density and cumulative distribution functions for any random variable X, respectively. Hence,  

 

(1)

 

Models of SDT generally assume Gaussian noise. Thus, for a target with tilt θ (where θ<0 indicates counterclockwise tilts and θ>0 clockwise), we have and , where φ(z) is the standard normal density function. Substituting these values into the previous equation we get  

 

(11)

where Φ(z) is the standard normal cumulative distribution function. For each fit, the parameter σ was allowed to vary between observers and durations, but not conditions. 

Assuming that the observer simply reports the mean apparent orientation (the red line in Figure 8), the probability of a correct identification under SDT is simply  

 

(12)

 

Assuming that the observer reports the orientation of the element having the greatest apparent tilt (the green line in Figure 8), it is slightly more complicated to derive the probability of a correct identification P_Max(I). Let E₁ denote the event that the actual target is tilted clockwise of horizontal. Assuming the observer has no response bias,  

 

(13)

 

Because the greatest apparent tilt can come from any distractor as well as the target, we have  

 

(14)

 

The ideal observer selects the most likely orientation (clockwise or counterclockwise), given all possible events E_i,j. Here, the integers i and j represent the tilt and position of the target. On any trial in the uncued condition, the target could assume 1 of 10 possible tilts, thus and, with 8 possible positions, i≤j≤8. Because the ideal observer has no response bias and , the probability of a correct identification is  

 

(15)

 

Let the vector represent the sensations arising from each position on a given trial. We need to determine how many of the possible sensations are more likely under the null hypothesis H₀: E_i,j, i>0 than under the alternative H₁: E_i,j, i<0. Thus  

 

(16)

where H(x) is the (Heaviside) unit-step function:  

 

(17)

 

Let f_i(u) denote the probability density function of apparent tilts produced by target i and let f₀(u) denote the probability density function of apparent tilts produced by a distractor. Therefore,  

 

(18)

 

In order to calculate the threshold ratio for the super-ideal observer (the blue line in Figure 8), we assumed that each target had 1 of only 2 possible tilts, ie . The preceding equation was approximated (by evaluating the integrand at 10⁷ points within the 8-dimensional hypercube with sides stretching from −3.5σ to 3.5σ along each dimension) iteratively until we found the θ₀ for which P_Ideal(I) = 3/4 when σ = 1.