Does precision decrease with set size?

Abstract

Abstract

Abstract:

Abstract  Thebrain encodes visual information with limited precision. Contradictory evidence exists as to whether the precision with which an item is encoded depends on the number of stimuli in a display (set size). Some studies have found evidence that precision decreases with set size, but others have reported constant precision. These groups of studies differed in two ways. The studies that reported a decrease used displays with heterogeneous stimuli and tasks with a short-term memory component, while the ones that reported constancy used homogeneous stimuli and tasks that did not require short-term memory. To disentangle the effects of heterogeneity and short-memory involvement, we conducted two main experiments. In Experiment 1, stimuli were heterogeneous, and we compared a condition in which target identity was revealed before the stimulus display with one in which it was revealed afterward. In Experiment 2, target identity was fixed, and we compared heterogeneous and homogeneous distractor conditions. In both experiments, we compared an optimal-observer model in which precision is constant with set size with one in which it depends on set size. We found that precision decreases with set size when the distractors are heterogeneous, regardless of whether short-term memory is involved, but not when it is homogeneous. This suggests that heterogeneity, not short-term memory, is the critical factor. In addition, we found that precision exhibits variability across items and trials, which may partly be caused by attentional fluctuations.

Introduction

The internal representation of visual information is noisy. As a consequence, the observer's measurement (best guess) of a task-relevant stimulus variable is also noisy. The precision of this measurement depends on many factors, such as stimulus contrast, presentation time, and retinal eccentricity. In this paper, we examine its dependence on the number of items in a visual scene. If the brain has a fixed amount of resource (e.g., measurements, energy, or spikes) to spend on encoding a scene, one would expect that precision decreases with the number of relevant items (set size). Specifically, if one were to distribute M measurements over N items (M typically being a very large number), and each individual measurement came with variance σ², then the variance by which an item is encoded would be Nσ²/M, and thus encoding precision or inverse variance would be inversely proportional to set size (Shaw, 1980). (A more continuous version of this argument relies on Fisher information; see Models.) An understanding of whether and how encoding precision depends on the number of relevant items in a scene would have direct implications for many areas of vision research, including multiple-object tracking, visual working memory, peripheral vision, and selective attention, and might be applied to real-world attentionally demanding tasks. 

In a series of pioneering experiments in the 1990s, John Palmer and colleagues discovered that precision indeed decreases with set size in some tasks (Palmer, 1990), but is constant in others (Palmer, 1994; Palmer, Ames, & Lindsey, 1993; Palmer, Verghese, & Pavel, 2000). Later, Wilken and Ma found that precision decreases with set size in change detection and delayed estimation (Wilken & Ma, 2004). Relevant studies are summarized in Table 1. We restrict ourselves here to tasks with a single relevant stimulus feature and a single target object. 

Table 1

 

Comparison of visual search and visual short-term memory studies with simple stimuli and a single target. Notes: (*) Also reported in Baldassi & Verghese, 2002. (**) This result should be interpreted with care, because none of the models provided a good fit. (***) This was a multiple-object tracking task. The degree of memory involvement is unclear.

Study	Task	Distractor distribution	Memory task	Effect of set size on precision
Palmer, 1990	Discrimination	Heterogeneous	Yes	Yes
Palmer et al., 1993	Visual search	Homogeneous	No	No
Palmer, 1994	Visual search	Homogeneous	No	No
Baldassi & Burr, 2000	Classification	Homogeneous	No	No (*)
	Localization	Homogenous	No	No
Wilken & Ma, 2004	Change detection	Heterogeneous	Yes	Yes
	Delayed estimation	Heterogeneous	Yes	Yes
Baldassi & Burr, 2006	Estimation	Homogeneous	No	No
Busey & Palmer, 2008	Visual search	Homogeneous	No	No
	Localization	Homogeneous	No	No (**)
Ma & Huang, 2009	Change discrimination	Heterogeneous	Yes (***)	Yes

We observe that the studies that have reported an effect of set size on precision all used heterogeneous distractors (distractors that differ amongst each other in the stimulus feature of interest) and a task that required subjects to encode a set of items in short-term memory. For example, Palmer (Palmer, 1990) found an effect of set size on precision in a paradigm in which observers were presented with a brief display consisting of horizontal lines of various lengths, followed, after a 2-second blank, by a display containing one line in one of the previous locations. Observers reported whether that line was longer or shorter than the corresponding line in the memorized display. The studies that reported constancy used homogeneous distractors (distractors identical to each other in the stimulus feature of interest) and tasks that did not involve a memory component. For example, Palmer (Palmer, 1994) found constancy when observers judged whether or not one of several discs had a higher contrast than the others. The purpose of the present paper is to determine whether heterogeneity or short-term memory involvement is the critical factor in producing a dependence of precision on set size. Wilken and Ma (Wilken & Ma, 2004) suggested that heterogeneity is critical, whereas Palmer (Palmer et al., 1993; Palmer, 1990) proposed that memory requirements are a likely source of set size effects. Resolving this debate will contribute to answering the question to what extent visual processing is resource-limited. 

The tasks used in this paper all require the observer to detect, based on orientation only, whether or not a single target object is present among a set of objects. It is very difficult to design such a target detection task with homogeneous distractors requiring short-term memory, as the target would stand out by its oddity, rendering memory involvement unnecessary. However, search with heterogeneous distractors and without a short-term memory requirement is easily realized and will be the critical condition of this paper. If short-term memory involvement causes precision to decrease with set size, we expect no effect of set size. If heterogeneity causes precision to decrease with set size, we expect a decrease. 

Of course, we are not the first to test human visual search with heterogeneous distractors. However, previous studies on this topic cannot answer our central question. One study focused on the effect of the type of heterogeneity, not that of set size (Rosenholtz, 2001). Precision was estimated at only two set sizes and with a different group of (very few) subjects at each. Other studies that used heterogeneous distractors did not vary set size within the same subjects (Ma, Navalpakkam, Beck, Van den Berg, & Pouget, 2011; Vincent, Baddeley, Troscianko, & Gilchrist, 2009), did not fit any models (Duncan & Humphreys, 1989), or did not test for an effect of set size on precision (Baldassi & Verghese, 2002). 

We conducted three experiments. In Experiment 1, subjects performed target detection with heterogeneous distractors under two conditions. In the precue condition, the target identity was revealed to the subject 1 second prior to the onset of the search display; in the postcue condition, the target identity was revealed 1 second after viewing the display, requiring subjects to memorize the entire display. If memory is the determining factor, we expect to find that precision decreases with set size in the postcue but not in the precue condition. If, on the other hand, display heterogeneity is crucial, we expect that precision decreases with set size in both conditions. In Experiment 2, we compared heterogeneous and homogeneous distractor conditions in a search task that did not require memorization of items. If memory is the determining factor, we expect to find that precision is constant with set size in both conditions. If display heterogeneity is crucial, we expect that precision decreases with set size in the heterogeneous but not in the homogeneous condition. Experiment 3 was a control experiment using homogeneous distractors. 

A closely related question that our experiments can shed light on is whether precision is equal across locations and trials. Models of perception routinely assume that this is the case, i.e., that the precision with which a stimulus is encoded is constant as long as the experimental conditions are kept fixed (Green & Swets, 1966). However, there are many factors that could make precision variable over space and time, such as stimulus dependencies (Girshick, Landy, & Simoncelli, 2011), fluctuations in intrinsic alertness (Matthias et al., 2009), higher-order configural effects (Brady & Tenenbaum, 2010), and covert shifts of attention (Pestilli & Carrasco, 2005; Posner, 1980). Considering these factors, it may be more proper to model precision itself as a random variable rather than assuming it to be fixed (Van den Berg, Shin, Chou, George, & Ma, 2012). To determine whether precision is subject to fluctuations, we also tested models in which mean precision is constant with set size or dependent on set size, but the actual precision for each stimulus is drawn from a probability distribution around that mean. To anticipate our results, we find that search under heterogeneous distractors is best described by the model in which precision fluctuates, and mean precision decreases with set size. 

Models

Task

The observer reports whether a target is present among a set of distractors (Figure 1a). Stimuli are characterized by their orientations, the target orientation is s_T, and the probability that the target is present equals 0.5. The target orientation is specified through a precue (Figure 1a, left) or a postcue (Figure 1a, right). When the target is present, its location is chosen randomly. Set size varies from trial to trial. 

Figure 1

 

Experiment 1. (a) Time course of a trial in the precue (left) and postcue (right) conditions. (b) Hit and false-alarm rates as functions of set size in the precue (left) and postcue (right) conditions.

Optimal-observer theory

The optimal-observer theory of visual search is well known both for homogeneous distractors (Peterson, Birdsall, & Fox, 1954) and for heterogeneous distractors drawn from a uniform distribution (Ma et al., 2011). We summarize it briefly here. The generative model of the task is illustrated in Figure 2. Target presence is denoted C and takes values 0 and 1. Target presence at the i^th location is denoted T_i and also takes values 0 and 1. The observer has access to noisy measurements, x = (x₁, … , x_N), of the stimuli, s = (s₁, … , s_N), and infers whether or not a target was present. For convenience, we remap all orientations from (−π/2, π/2) to (−π, π) in our models and analyses. The measurement of the i^th stimulus, x_i, follows a Von Mises (circular normal) distribution centered at the true stimulus orientation, s_i:  

 

where the concentration parameter, κ_i, is related to precision (see following), and I₀ is the modified Bessel function of the first kind of order 0. For large κ, a Von Mises distribution is accurately approximated by a Gaussian distribution with σ² = 1/κ. We will use Gaussian distributions in the case of homogeneous distractors because, there, stimuli and measurements are all concentrated in such a small part of the circular space that the space can be treated as a line. 

Figure 2

 

Optimal-observer model. (a) Generative model. Arrows indicate dependencies between variables. C indicates whether the target is present (0 or 1), T target presence at each location (a vector of 0s with possibly one 1), s the vector of stimulus orientations, and x the vector of internal measurements. (b) Inference process. The optimal observer estimates C from x by comparing p(C = 0|x) to p(C = 1|x). This consists of first computing local evidence about the presence of a target at each location, d_i, and then combining the resulting N numbers into a global decision variable, d. The sign of d determines Ĉ, the estimate of C.

On each trial, the optimal observer computes the posterior probability that the target is present given the measurements, denoted p(C = 1|x), and reports “target present” if this probability is greater than 0.5. This is equivalent to reporting “target present” when the log posterior ratio, denoted d, is positive. To compute the log posterior ratio, we first apply Bayes' rule:  

 

where p_present is the observer's prior probability that the target is present. (This does not have to be equal to 0.5, the true frequency of target presence.) The likelihood function of C, p(x|C), is computed by marginalizing over both s and T = (T₁, … , T_N). After some basic math, we find  

 

where d_i is defined as  

 

The relationship between d and d_i in Equation 2 would be different if distractor orientations were not drawn independently or if more than a single target could be present. When distractors are heterogeneous and drawn from a uniform distribution, the distractor distribution is p(s_i|T_i = 0) = 1/(2π). Using this expression as well as Equation 1, Equation 3 becomes (Ma et al., 2011)  

 

When distractors are homogeneous with an orientation equal to s_D, we use Gaussian distributions, and Equation 3 becomes (Peterson et al., 1954)  

 

We obtained the predictions of the model for an individual trial by drawing 10,000 sets of N measurements each from Von Mises (or Gaussian) distributions centered on the respective stimuli on that trial and applying the decision rule to each set of measurements. This results in a predicted probability that the subject will report “target present” on that trial, $p\left(Ĉ|s,\theta \right)$, where θ denotes the model parameters. 

Relationship between κ and precision

To specify the relation between encoding precision and κ, we identify encoding precision with Fisher information, J(s), which measures the best possible decoder performance based on the neural activity encoding the stimulus (Paradiso, 1988; Seung & Sompolinsky, 1993). Fisher information is under the general condition of Poisson-like variability proportional to the amplitude of the population activity encoding s (Ma, Beck, Latham, & Pouget, 2006). Fisher information for a noise distribution p(x|s) is defined as  

 

where the expected value 〈s〉 is over the noise distribution p(x|s). Substituting Equation 1, we find that J(s) is independent of s and equal to  

 

where I₁ is the modified Bessel function of the first kind of order 1. Equation 5 states a general relationship between the precision with which a stimulus is encoded, J, and the concentration parameter, κ. Note that J is a monotonically increasing function of κ and therefore invertible. The equivalent relationship for Gaussian noise is J = 1/σ² (Palmer, 1990; Shaw, 1980). 

Flat versus nonparametric models

To determine whether encoding precision depends on set size, we will compare a model in which encoding precision is constant (“flat”) against one in which precision is allowed to vary across set sizes. In the latter model, to avoid making a potentially wrong assumption about the relationship between encoding precision and set size, we will fit encoding precision separately for each set size (“nonparametrically”). 

Equal-precision versus variable-precision models

The two models previously described assume that encoding precision is equal across items in a display and across trials with the same set size. We therefore call them “equal-precision” (EP) models. However, considering factors such as attentional fluctuations, one might expect that encoding precision varies across items and trials. To allow for this possibility, we also include “variable-precision” (VP) variants of the flat and nonparametric models (Van den Berg et al., 2012). In the VP models, the precision with which an item is encoded is a random variable. Its expected value, denoted $J$, follows a flat or nonparametric relationship with set size. We model fluctuations by drawing precision, independently for each item and each trial, from a gamma distribution with mean $J$ and scale parameter τ. 

Summary of models

We end up with four models: 

•  
  FlatEP (flat, equal precision): Precision does not fluctuate or depend on set size. This model has two free parameters (p_present and $J$).
•  
  FlatVP (flat, variable precision): Precision fluctuates but does not depend on set size. This model has three free parameters (p_present, $J$, and τ).
•  
  npEP (nonparametric, equal precision): Precision does not fluctuate but may depend on set size. This model has five free parameters (p_present and one value of $J$ for each set size).
•  
  npVP (nonparametric, variable precision): Precision fluctuates and may depend on set size. This model has six free parameters (p_present, τ, and one value of $J$ for each set size).

Methods

Experiment 1

Subjects

Initially, 14 subjects, including the three authors, performed the experiment (11 female, 3 male). All subjects had normal or corrected-to-normal acuity. After the first session, subjects with an overall performance of less than 65% were excluded from further participation. Two subjects met this criterion with performances of 63% and 59%. The remaining 12 subjects had a first-session performance of 71.9 ± 0.8% and completed the entire experiment. 

Apparatus and stimuli

Stimuli were presented on a 21″ LCD monitor with a refresh rate of 60 Hz. Subjects viewed the displays from a distance of approximately 60 cm. Background luminance was 33.6 cd/m². Each stimulus was a Gabor patch with a spatial frequency of 1.05 cycles/degree, a standard deviation of 0.52°, and a peak luminance of 132 cd/m². The relevant stimulus feature was orientation. 

On each trial, target and distractor orientations were drawn independently from a uniform distribution on (−90°, 90°), and there was a 50% chance that the target was present in the search display. Subjects were informed of this probability in advance. The number of stimuli in a search display was one, two, three, or four (randomly interleaved). Stimuli were spaced at 90° angular intervals on an imaginary circle of radius 5° around the center of the screen with a random overall rotation. 

Procedure for precue condition

Each trial began with subjects fixating on a central cross (200 ms) followed by a single stimulus with the orientation of the target (117 ms), a blank screen (1 s), and a search display (117 ms) (Figure 1a, left). On a target-present trial, the location of the target was chosen randomly from the stimulus locations. Subjects reported whether they believed the target was present in the search display through a key press. Correctness feedback was given by turning the fixation cross green (correct) or red (incorrect) during the 500-ms intertrial period. 

Postcue condition

The postcue condition was identical to the precue condition except that the temporal order of the cue and the search display was reversed (Figure 1a, right). 

Blocks and sessions

The experiment was conducted over four sessions on different days. Each session consisted of three precue and three postcue blocks presented in random order. Each block consisted of 125 trials. After each block, the percentage correct in that block was shown to the subject along with a high-score ranking. Each subject completed 3,000 trials in total. 

Experiment 2

Experiment 2 tested, in the absence of memory involvement, whether homogeneity or heterogeneity of the distractors would affect the dependence of precision on set size. Differences with Experiment 1 were as follows. Initially, 12 subjects, including the three authors, performed the experiment (3 female, 9 male). Three subjects met the exclusion criterion with first-session performances of 61%, 61%, and 59%. The remaining 9 subjects had a first-session performance of 79.5 ± 1.3% and completed the entire experiment. The heterogeneous condition was identical to the precue condition of Experiment 1 except that the target was always vertical instead of being drawn from a uniform distribution. To remind the subject, the target orientation was still shown as a precue on each trial. Fixing the target orientation was not possible in Experiment 1 as the postcue condition would then not have required memory. Set size was 2, 4, 6, or 8. Stimuli were spaced at 45° angular intervals on an imaginary circle of radius 5° around the center of the screen. The homogeneous condition was identical to the heterogeneous condition except that distractors had a fixed orientation of 5° clockwise with respect to vertical. Each session consisted of three heterogeneous and three homogeneous blocks, randomly interleaved. 

Experiment 3

Experiment 3 served as a control experiment for the homogeneous condition of Experiment 2 to ensure that fixing target and distractor orientations throughout the experiment was not critical. Experiment 3 was identical to the precue condition of Experiment 1 (importantly, the target varied from trial to trial) except for the following differences. Five subjects, including one author, performed the experiment (3 female, 2 male). Distractors were homogeneous with an orientation 10° clockwise with respect to the target orientation. Subjects completed two sessions, each consisting of six blocks, each consisting of 125 trials, for a total of 1,500 trials. 

Results

Experiment 1

In both the precue and postcue conditions, performance decreases as a function of set size (Figure 1b). Specifically, hit rates decrease (one-way repeated-measures ANOVA for precue: F(3, 33) = 24.0, p < 0.001; postcue: F(3, 33) = 56.0, p < 0.001) and false-alarm rates increase (precue: F(3, 33) = 189.1, p < 0.001; postcue: F(3, 33) = 79.4, p < 0.001). 

We used maximum-likelihood estimation (Appendix A) to fit the four models to the data. Parameter estimates are given in Appendix B. The fits to the hit and false-alarm rates are shown in Figure 3. Differences between the models are subtle but noticeable with the npVP model fitting best. The root mean squared error between model and data across all subjects, all set sizes, and both target presence conditions (RMSE) was 0.037 for precue and 0.049 for postcue. 

Figure 3

 

Hit and false-alarm rates in Experiment 1 for the four models (rows). Throughout the paper, error bars and shaded areas represent one standard error of the mean for subject data and model fits, respectively, and values in plots indicate RMSEs.

To examine the data in greater detail, we plotted the proportion of “target present” responses as a function of the smallest circular difference between the target orientation and any of the distractor orientations, separately for target-present and target-absent trials, in both the precue (Figure 4) and postcue (Figure 5) conditions. As one would expect, the proportion of “target present” responses increases as the closest distractor becomes more similar to the target. Fits of the four models to these psychometric curves are shown in Figures 4 and 5 as well. The npVP model provides the best fits (RMSE was 0.11 for precue and 0.12 for postcue). 

Figure 4

 

Psychometric curves and model fits in the precue condition of Experiment 1. Proportion “target present” responses as a function of the minimum circular difference between the target orientation and any of the distractor orientations, separately for target-present (blue) and target-absent (red) trials, for each set size (columns) and model (rows). For set size 1, this difference does not exist on target-present trials.

Figure 5

 

Psychometric curves and model fits in the postcue condition of Experiment 1. See caption of Figure 4 for details.

To compare the models more rigorously, we performed Bayesian model comparison (see Appendix A for details). Rather than being based on summary statistics, this method takes into account the specific orientations presented on each individual trial by computing the probability of the subject response given the model and those orientations. It automatically accounts for differences between the numbers of free parameters in the models (MacKay, 2003). In Bayesian model comparison, the npVP model outperforms the flatEP, flatVP, and npEP models by 14.2 ± 3.7, 7.2 ± 3.6, and 14.2 ± 2.6 log likelihood points in the precue condition, and by 38.7 ± 5.8, 19.1 ± 3.9, and 31.7 ± 4.2 log likelihood points in the postcue condition (Figure 6). A difference of 7.2 log likelihood points means that the model is exp(7.2) = 1,339 times more likely. 

Figure 6

 

Bayesian model comparison for Experiment 1. Log likelihood of the flatEP, flatVP, and npEP models minus that of the npVP model for each subject (left) and averaged over subjects (right) in the precue (top) and postcue (bottom) conditions.

The finding that the npVP model outperforms the other models by large margins indicates that encoding precision depends on set size and is subject to variability. The relationship between mean encoding precision and set size is well captured by a power law function in both conditions (Figure 7a). In the precue condition, we find a power of −0.74 ± 0.12; in the postcue condition, we find a power of −0.98 ± 0.10. These powers are significantly different from 0 (precue: t(11) = −6.07, p < 0.001; postcue: t(11) = −10.20, p < 0.001), confirming that mean encoding precision decreases with set size, both in the precue and in the postcue conditions. Put differently, the standard deviation of the noise increases with set size (Figure 7b). Standard deviation was computed as  

 

 

Figure 7

 

Dependence of precision and noise level on set size in Experiment 1. (a) Estimates of mean precision (mean over items and trials, not over subjects) at each set size in the npVP model in the precue (search) and postcue (memory) conditions. Lines represent the mean best-fitting power laws (mean over subjects). (b) As (a), but for the standard deviation of the noise, computed from the mean precision for each subject. The power law fits are inherited from (a).

Experiment 2

The results of Experiment 1 suggest that precision decreases with set size when distractors are heterogeneous, regardless of whether short-term memory is required in the task. In Experiment 2, we test the hypothesis that precision decreases with set size when distractors are heterogeneous but not when they are homogeneous. No short-term memory is involved in either the heterogeneous- or the homogeneous-distractor condition of Experiment 2. A difference between the precue condition in Experiment 1 and the heterogeneous-distractor condition in Experiment 2 is that, in the latter, the target was always vertical, thus further reducing the influence of memory. 

Hit rates in the heterogeneous condition decreases (F(3, 24) = 29.1, p < 0.001) and false-alarm rates increases (F(3, 24) = 56.8, p < 0.001) as a function of set size (Figure 8a). The npVP model provides a slightly better fit than the other models. RMSE values are 0.055, 0.057, 0.046, and 0.040 for the flatEP, flatVP, npEP, and npVP models, respectively. 

Figure 8

 

Experiment 2, heterogeneous condition. (a) Hit and false-alarm rates. The shaded areas show the fits of the npVP model. (b) Proportion “target present” responses as a function of the minimum circular difference between the target orientation and any of the distractor orientations, separately for target-present (blue) and target-absent (red) trials, for each set size (panels). The shaded areas show the fits of the npVP model. (c) Bayesian model comparison results for each subject (left) and average over subjects (right).

The npVP model provides a good fit to the psychometric curves (Figure 8b) and outperforms other models (RMSE values are 0.14, 0.12, 0.13, and 0.11 for the flatEP, flatVP, npEP, and npVP models, respectively). Bayesian model comparison shows that the data are most likely under the npVP model (Figure 8c); the log likelihood differences between the npVP model and the flatEP, flatVP, and npEP models were 50 ± 15, 13.8 ± 5.1, and 48 ± 14, respectively. The relationship between mean precision and set size in the npVP model is captured well by a power law function (Figure 9a),with a power of −0.73 ± 0.14. This is consistent with the precue condition of Experiment 1. The scale is different though; this might be due to the fact that the target orientation is fixed across trials. 

Figure 9

 

Dependence of precision and noise level on set size in Experiment 2. (a) Estimates of mean precision (mean over items and trials, not over subjects) at each set size in the npVP model in the heterogeneous condition (blue points) and in the flatVP model in the homogeneous condition (red). The blue line represents the mean best-fitting power law (mean over subjects). (b) As (a), but for the standard deviation of the noise, computed from the mean precision for each subject. The power law fit is inherited from (a).

Hit rate in the homogeneous condition decreases with set size (F(3, 24) = 20.58, p < 0.001) but there is no significant effect of set size on the false-alarm rate (F(3, 24) = 1.54, p = 0.23; Figure 10). Unlike Experiment 1 and the heterogeneous condition of Experiment 2, where the random drawing of the target and/or distractor orientations produced a unique set of stimuli on each trial, the homogeneous condition of Experiment 2 had only eight different trial types (target present or absent at four set sizes). Because the npEP and npVP models have five and six free parameters, respectively, fitting these models to the data from the homogeneous condition would likely result in overfitting. Therefore, we replaced these models with variants in which (mean) encoding precision was related to set size by a power law function: $J\left(N\right)=J_1{N}^{\alpha }$ (plEP model) or $J\left(N\right)=J_1{N}^{\alpha }$ (plVP model). For each subject, the value of α was taken from the analysis in Figure 9a (heterogeneous condition). Also, we fixed the prior to 0.5. 

Figure 10

 

Hit and false-alarm rates in Experiment 2, homogeneous condition with model fits. In the PL models, the (mean) precision per set size followed a power law with the power obtained per subject from the heterogeneous condition (Figure 9a, blue).

The “flat” models fitted the hit and false-alarm rates better than the power law models (Figure 10). Furthermore, Bayesian model comparison revealed that the flatVP model provides the most likely description of the data; the differences in log likelihood with the flatEP, plEP, and plVP models were 1.8 ± 1.3, 13.8 ± 3.2, and 12.0 ± 3.4, respectively. This means that, in the homogeneous condition, we cannot distinguish between equal and variable precision, but (mean) precision tends to be independent of set size. We also fitted a general power law as discussed previously but with α being a free parameter instead of being determined by the heterogeneous condition. In the plEP model, we found a power of −0.08 ± 0.09 and in the plVP model a power of −0.10 ± 0.08. Neither is significantly different from zero (t(8) = −0.87, p = 0.41; t(8) = −1.28, p = 0.24, respectively). Our results indicate that the mean encoding precision decreases with set size in the heterogeneous condition but is constant with set size in the homogeneous condition (Figure 9a). 

Experiment 3

An alternative explanation for the difference in the dependence of mean precision on set size between conditions could be that, in the homogeneous condition, the distractor orientation was identical for every trial and could therefore be learned. To determine whether this explanation was valid, we performed a control experiment (Experiment 3) in which, on each trial, the target was drawn from a uniform distribution and distractors were homogeneous with an orientation 10° clockwise with respect to the target. We compared “flat” models (EP and VP) against power law models (EP and VP) with power equal to −0.74 (mean power from npVP model in the precue condition in Experiment 1). As in the homogeneous condition of Experiment 2, the “flat” models fitted the hit and false-alarm rates better than the power law models (Figure 11). Bayesian model comparison revealed that the flatVP model provides the most likely description of the data. The differences in log likelihood with the flatEP, plEP, and plVP models were 1.4 ± 1.5, 12.1 ± 4.2, and 11.5 ± 4.1, respectively. 

Figure 11

 

Hit and false-alarm rates in Experiment 3 with model fits. In the PL models, the (mean) precision per set size followed a power law with the power obtained from Experiment 1, precue condition (Figure 7a, blue).

We also fitted a general power law with the power α being a free parameter instead of being determined by the precue condition of Experiment 1. In the plEP model, we found a power of −0.16 ± 0.10 and in the plVP model a power of −0.20 ± 0.07. Although not significantly different from zero (t(4) = −1.57, p = 0.19; t(4) = −2.83, p = 0.048, respectively), these powers seem slightly more negative than in the heterogeneous condition of Experiment 2. 

In summary, the standard deviation of the noise increases with set size in the heterogeneous condition, but is constant in the homogeneous condition (Figure 9b). This supports the hypothesis that stimulus heterogeneity is the key factor determining whether encoding precision decreases with set size. 

Apparent guessing

Our results suggest that encoding precision in the heterogeneous condition is subject to random fluctuations. While the EP models perform reasonably well in terms of fitting the summary statistics (Figures 4 and 5), they lose by large margins in the model comparison (Figures 6 and 8c). An important difference between the EP and VP models is that, in the latter, items are frequently encoded with such a low precision that it may appear as if the subject is making a random guess about the orientation of the stimulus. The distribution of precision, shown in Figure 12a with the fitted parameters of one subjects, contains considerable probability mass near zero, especially at higher set sizes. To test whether the absence of such “apparent guessing” in the EP models can explain why they perform so poorly, we fitted the subject data using a variant of the npEP model with a set-size-dependent guessing rate, i.e., an npEP model in which the subjects gave a random response on a certain proportion of trials. We found that the estimated guessing rate is significantly higher than zero at all set sizes (t(11) > 2.7, p < 0.02 and t(11) > 2.3, p < 0.05 for precue and postcue conditions of Experiment 1, respectively, and t(11) > 2.33, p < 0.05 for heterogeneous condition of Experiment 2) and increases with set size (F(3, 33) = 11.33, p < 0.001, F(3, 33) = 34.42, p < 0.001, and F(3, 24) = 7.22, p < 0.001, respectively). This increase is reminiscent of that found in visual short-term memory studies (Bays, Catalao, & Husain, 2009; Zhang & Luck, 2008; Van den Berg et al., 2012). Here, we argue that this guessing is only apparent and accounted for by the npVP model. 

Figure 12

 

The distribution of precision in the VP models can explain apparent guessing rates. (a) Example precision distribution based on the ML parameter estimates of subject TS in Experiment 1, precue condition. (b) Apparent guessing rates were obtained by fitting an npEP model extended with a set size-dependent guessing rate to subject data (circles) and to simulated npVP (blue) and simulated npVPG (red) observers with parameters estimated from subject data (shaded areas). In an EP model, the apparent guessing rate would be zero at every set size.

For each subject, we computed the apparent guessing rate predicted by the npVP model by generating synthetic npVP data using that subject's maximum-likelihood estimates of the model parameters obtained earlier and then applying the apparent-guessing rate analysis to these synthetic data. The npVP model reproduces the increase of the apparent guessing rate with set size qualitatively across all three relevant experimental conditions but slightly underestimates its magnitude (Figure 12b). The npVP model can account for the increasing trend because, as set size increases, the distribution of precision gets pushed more and more toward zero (Figure 12a). The npEP model with a set-size-dependent guessing rate in essence approximates the continuous distributions over precision in the npVP model by a discrete distribution on two values of precision: zero (guessing) and one nonzero value (the constant precision). Near-zero values in the npVP distribution end up being lumped into the guessing component. 

We also tested whether an addition of a constant guessing rate to the npVP model would improve its prediction for the apparent guessing rate. We call this model the npVPG model. The npVPG model accounts for the apparent guessing rate slightly better (Figure 12b), and performs about as well in Bayesian model comparison (log likelihood differences relative to npVP: 2.4 ± 1.7 in Experiment 1 precue, 1.5 ± 1.1 in Experiment postcue, and 2.12 ± 0.93 in Experiment 2). These results provide weak evidence for the presence of a constant guessing rate. Guessing rate was estimated to be 0.048 ± 0.016 in Experiment 1 precue, 0.054 ± 0.015 in Experiment 1 postcue, and 0.049 ± 0.015 in Experiment 2. 

The npEP model with set-size-dependent guessing still performs worse than the npVP model in accounting for subject data. The differences in log likelihood difference are 4.3 ± 2.8 and 7.7 ± 3.0 in the precue and postcue conditions of Experiment 1 and 2.9 ± 2.6 in the heterogeneous condition of Experiment 2. This suggests that the distribution over precision is continuous as in Figure 12a instead of a mixture of constant precision and set-size-dependent guessing. 

Discussion

Using three visual search experiments, we examined whether stimulus heterogeneity or short-term memory involvement determines the dependence of the precision with which visual stimuli are encoded on set size. In Experiment 1, we found that precision decreases with set size when the distractors are heterogeneous regardless of whether or not the search display had to be memorized. This finding suggests that short-term memory is not the factor determining whether encoding precision depends on set size. In Experiment 2, using a task that did not involve short-term memory, we found that precision is constant across set sizes when distractors are homogeneous but decreases when distractors are heterogeneous. Constancy under distractor homogeneity was confirmed in Experiment 3. Combining the results from the three experiments, we conclude that stimulus heterogeneity, not short-term memory involvement, is likely to be the key factor determining the relationship between encoding precision and set size. This confirms a speculation by Wilken and Ma (Wilken & Ma, 2004). By contrast, we do not find evidence for Palmer's suggestion that, for simple stimuli, memory requirements determine whether or not precision depends on set size (Palmer, 1990; Palmer et al., 1993). In none of our experiments did we needed to assume a limited item capacity (Cowan, 2001) to describe the data well. 

It is intriguing that precision seems to depend on set size only when distractors are heterogeneous. This could be because homogeneous displays are special; they contain high-level structure in the form of the sameness of all distractors. It might be that specialized neurons—for instance, ones tuned to texture or parallelism—detect such sameness and compute in a manner that does not require scrutiny of every individual item. This would allow the brain to bypass resource limitations associated with focused attention. By contrast, the heterogeneous condition does not contain any structural cues and would require scrutiny. At a more mechanistic level, this view is broadly consistent with Hochstein and Ahissar's Reverse Hierarchy Theory (Hochstein & Ahissar, 2002) in which rapid feedforward processing serves to extract categorical or gist information from a scene, while feedback processing recovers details. This theory might predict that search with homogeneous distractors, but not with heterogeneous, would benefit from feed-forward processing. Further work should examine whether the degree of heterogeneity affects the relationship between encoding precision and set size. 

Another finding of the present study is that, apart from depending on set size, encoding precision is also subject to random fluctuations, at least when the distractors are heterogeneous. Bayesian model comparison showed that the models in which precision varies across items and trials provide more likely descriptions of the heterogeneous-distractor data than the models in which precision is fixed. (For homogeneous distractors, this analysis was inconclusive.) We also showed that the variable-precision model accounts for both the presence of what seems to be guessing and the increase of the apparent guessing rate with set size (Figure 12b). These results are consistent with recent ones in short-term memory tasks (Van den Berg et al., 2012). According to the variable-precision model, the finding of a nonzero guessing rate is a side effect arising from wrongly assuming equal precision. We speculate that fluctuations in precision partly reflect temporal and spatial fluctuations in attention. Variability in encoding precision has implications for models of perception, which routinely assume that precision is fixed. 

When plotting the proportion of “target present” responses as a function of the minimum circular difference between the target and any of the distractors (Figures 4, 5, and 8b), we observe a dip in the curve for the “target present” trials (most notably for N = 8 in the heterogeneous condition of Experiment 2; Figure 8b). None of the models was able to account for this dip. A speculative account of this effect could be that search proceeds in two stages, again somewhat in the vein of Reverse Hierarchy Theory. In the first stage, orientations close to the target orientation are identified, while in the second stage, attentional resources are deployed to those selected orientations. Then, as the distractors collectively become more dissimilar to the target, the number of items that needs to be scrutinized decreases, thereby increasing performance. 

Several other directions suggest themselves. In the homogeneous condition of Experiment 2, we characterized human behavior using only eight numbers. This limits the complexity of the models one can test. The data set can be enriched in several ways. Furthermore, one could conduct the present study with more complex stimuli (such as those used in [Palmer, 1994]), replicate it in different feature dimensions, use discrimination instead of detection tasks, allow for the presence of multiple targets (such as [Wilken & Ma, 2004]), or exogenously manipulate attention.  

Acknowledgments

W.J.M. is supported by award number R01EY020958 from the National Eye Institute. R.v.d.B. is supported by the Netherlands Organisation for Scientific Research. The content of the manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the National Eye Institute or the National Institute of Health. We thank Nuwan de Silva for his idea about the two-stage process (see Discussion). 

Helga Mazyar and Ronald van den Berg contributed equally to this work. 

Commercial relationships: none. 

Corresponding author: Wei Ji Ma. 

Email: [e-mail]. 

Address: Baylor College of Medicine, Houston, Texas, USA. 

Appendix A

Model fitting and model comparison

Our model fitting and model comparison methods use each individual response instead of relying on summary statistics. Each model m produces a prediction about the response on each trial, p(Ĉ_k|s_k,θ), where Ĉ_k indicates the observer's response on trial k, s_k the presented set of stimuli, and θ the model parameters. The parameter likelihood function is the probability of finding a subject's actual responses under the model as a function of θ,  

 

where we assume that responses are conditionally independent across trials. Maximum-likelihood estimation consists of finding the parameters θ that maximize L. 

Bayesian model comparison (MacKay, 2003; Wasserman, 2000) is a principled method to compute the goodness of a model while penalizing extra free parameters. It consists of averaging the parameter likelihood over parameter space to find the probability of the data given the model:  

 

It is convenient to compute the log likelihood and write it as  

 

where

$L\left(\theta ,m\right)={\displaystyle \sum _{k=1}^{N_{\text{trials}}}\log p\left({Ĉ}_k|s_k,\theta \right)}$

and $L_{\max }\left(m\right)={\max }_{\theta }L\left(\theta ,m\right).$ This form prevents numerical problems, because the exponential in the integrand of Equation 6 is now of order 1 near the maximum-likelihood value of θ. For the j^th parameter, we assume a uniform distribution on an interval whose size we denote R_j. Then Equation 6 becomes  

 

where dim θ is the number of parameters. Intervals were (0.3, 0.7) for p_present, (0.5,100) for J in Experiment 1, (10, 200) for τ in Experiment 1, (5, 400) for J in Experiment 2, and (25, 500) for τ in Experiment 2. We approximated the integrals over parameters numerically by using the trapezoidal rule with 25 steps for p_present and 30 steps for all other parameters. Finally, log L(m) is compared between different models m. 

Appendix B

The following tables show the mean and standard error (across subjects) of the maximum-likelihood estimates of the parameters of each model for each experiment. 

Table A1

 

Parameter estimates in the flatEP model. Notes: (*) This value was set to 0.5.

Experiment	J	ppresent
1, Precue	6.52 ± 0.96	0.48 ± 0.01
1, Postcue	6.5 ± 1.1	0.44 ± 0.01
2, Heterogeneous	47 ± 11	0.42 ± 0.02
2, Homogeneous	235 ± 36	N/A (*)

Table A2

 

Parameter estimates in the flatVP model. Notes: (*) This value was set to 0.5.

Experiment	J	ppresent	τ
1, Precue	13.0 ± 1.7	0.53 ± 0.01	13.08 ± 0.82
1, Postcue	14.8 ± 2.8	0.49 ± 0.01	26.21 ± 5.0
2, Heterogeneous	135 ± 14	0.50 ± 0.03	277.58 ± 59
2, Homogeneous	256 ± 33	N/A (*)	40 ± 11

Table A3

 

Parameter estimates in the npEP model.

Experiment	J1	J2	J3	J4	ppresent
1, Precue	13.03 ± 2.37	6.84 ± 1.54	5.56 ± 0.94	5.64 ± 0.90	0.49 ± 0.01
1, Postcue	13.6 ± 2.6	6.2 ± 2.0	5.0 ± 1.2	4.9 ± 1.4	0.45 ± 0.01
2, Heterogeneous	88 ± 15	42 ± 12	44 ± 11	42 ± 10	0.42 ± 0.02

Table A4

 

Parameter estimates in the npVP model.

Experiment	J 1	J 2	J 3	J 4	ppresent	τ
1, Precue	25.9 ± 3.4	16.0 ± 2.7	11.8 ± 2.0	10.6 ± 2.0	0.54 ± 0.01	18.7 ± 2.9
1, Postcue	40.4 ± 7.1	21.9 ± 5.2	17.9 ± 4.2	12.2 ± 3.5	0.51 ± 0.01	51 ± 12
2, Heterogeneous	246 ± 27	138 ± 12	112 ± 11	92 ± 15	0.51 ± 0.03	296 ± 62