The dynamics of sensory buffers: Geometric, spatial, and experience-dependent shaping of iconic memory

Abstract

When a stimulus is presented, its sensory trace decays rapidly, lasting for approximately 1000 ms. This brief and labile memory, referred as iconic memory, serves as a buffer before information is transferred to working memory and executive control. Here we explored the effect of different factors—geometric, spatial, and experience—with respect to the access and the maintenance of information in iconic memory and the progressive distortion of this memory. We studied performance in a partial report paradigm, a design wherein recall of only part of a stimulus array is required. Subjects had to report the identity of a letter in a location that was cued in a variable delay after the stimulus onset. Performance decayed exponentially with time, and we studied the different parameters (time constant, zero-delay value, and decay amplitude) as a function of the different factors. We observed that experience (determined by letter frequency) affected the access to iconic memory but not the temporal decay constant. On the contrary, spatial position affected the temporal course of delay. The entropy of the error distribution increased with time reflecting a progressive morphological distortion of the iconic buffer. We discuss our results on the context of a model of information access to executive control and how it is affected by learning and attention.

Introduction

Multiple stimuli are continuously being processed in parallel by the sensory systems, most of which elicit only a brief transient sensory response that fades after few hundred milliseconds without reaching working memory, executive control, and consciousness. However, during this transient response, stimulus information is available to the system, and thus this brief transient response constitutes a memory that Neisser (1967) referred as iconic memory. This sensory buffer, which precedes the formation of short-term or working memory, was extensively studied by Sperling in the sixties, using the partial report paradigm (Sperling, 1960). Sperling showed that when observers saw briefly presented displays composed of several alphanumeric characters, only a few (3 to 5) elements could be remembered. This was consistent with the limits of short-term memory that had been known since, at least, the early experiments of Cattell (1886). However, observers had a much better memory when required to identify a specific subset of the characters at a short interval after the removal of the visual display (partial report). This indicated the existence of a high capacity initial memory of the stimulus display that decayed approximately 1000 ms after stimulus presentation. Since then, numerous studies have studied Iconic Memory (Coltheart, 1980; Loftus, Duncan, & Gehrig, 1992; Lu, Neuse, Madigan, & Dosher, 2005), addressing its characteristics such as their duration (Averbach & Sperling, 1961), content (Chow, 1986; Turvey & Kravetz, 1970), maintenance and extinction (Averbach & Coriell, 1961; Dember & Purcell, 1967), and models of information transfer to working memory (Gegenfurtner & Sperling, 1993; Loftus et al., 1992). The emergent picture from these studies is that iconic memory is extremely short (less than a second), has a great capacity of storage, and is labile (i.e., it can be disrupted by a competing stimulus). 

Inspired in this finding, different cognitive theories have proposed a two-stage model of access to consciousness or short-term memory (Chun & Potter, 1995; Dehaene, Sergent, & Changeux, 2003): the first stage involves an effortless parallel processing of multiple sensory elements and is available to the system only for a short-time. Only on a second stage, a subset of the iconic buffer is amplified, sustained, and broadcasted to be accessible for conscious processing. In this scheme, accessibility to the second stage is determined by three factors:

1.  
   the access of a stimulus to iconic memory of sensory receptors,
2.  
   the duration in iconic memory and
3.  
   the selection, i.e., the probability that an item of iconic memory is selected, retrieved, and amplified to short-term memory and conscious control.

Bottom-up (stimulus saliency) and top-down (task setting, attention) mechanisms as well as long-term plasticity are known to be involved in controlling this different stages (Awh, Vogel, & Oh, 2006; Cowan & Morey, 2006; Duncan, 1984; Gilbert & Sigman, 2007; Itti & Koch, 2001; Khayat, Spekreijse, & Roelfsema, 2006; Palmer, 1990; Roelfsema, Lamme, & Spekreijse, 1998; Schmidt, Vogel, Woodman, & Luck, 2002; Sigman & Gilbert, 2000; Woodman, Vecera, & Luck, 2003). The aim of the current work is to understand the dynamics of these stages in a partial-report paradigm experiment in which geometric, experiential, and attentional aspects are being manipulated. To asses this, we studied the effects of letter frequency (changes resulting from experience), geometry (saliency of the stimuli due to bottom-up morphological differences), and spatial location (sensibility to different strategies of allocation of attention in space) in the different aspects of the sensory buffer: its access, its maintenance, and the probability of being transferred to working memory. 

Results

Partial report paradigm

In each trial, subjects saw—while maintaining fixation in a cross at the center of the display—a circular eight-letter array (which excluded meridian locations) that lasted 106 ms ( Figure 1A). At a variable inter-stimulus interval (ISI) ranging from 24 to 1000 ms following the stimulus presentation, a small red circle was presented adjacent to a random location of the array, which indicated the identity of the letter that had to be responded. The cue was very small (12 times smaller than the average letter size) to minimize the possibility that they may induce masking of the target letters. The response was not speeded and subjects responded on a computer keyboard. 

Figure 1

 

Experimental design and principal factors affecting stimulus visibility. (A) Experimental design: A circular array of eight letters was presented during 106 ms. Subjects fixated in a cross at the center of the array. After a delay (which varied randomly between 0 and 1 s), a small red circle was presented in one of the eight locations of the array, and subjects had to type the letter presented in this position. (B) Temporal factor: time course of iconic memory. The proportion of correct responses ( p′) decays exponentially with ISI, as shown in previous studies. (C) Spatial factor: spatial-dependent performance in the memory task. Performance ( p′) as a function of spatial position is represented in a color map, shown on the left. Performance was overall better on the right visual field, close to the horizontal meridian, and worse in the bottom-left quadrant. (D) Form factor: performance for different letters, averaged across all positions, and ISI showed a broad dispersion. This difference was consistent across subjects. For instance, the letter “A” was the more visible letter for 10 out of 19 subjects.

Previous studies of iconic memory have for the most part studied the limits of iconic memory in highly practiced subjects. Here we performed a first study in which we examined iconic memory in a population of naïve subjects and observed that 19 out of 24 subjects showed a reliable decay of performance across time ( Figure 1B). Further analysis of Experiment I is based on these 19 subjects. In a second experiment, we will specifically study the iconic memory decay in practiced subjects, performing multiple experimental sessions, to determine quantitatively the parameters of the memory decay function. 

Principal factors affecting stimulus visibility

Before analyzing the effects of form (letter identity), space (position in the array), and experience (letter frequency) in the dynamics of iconic memory, we wanted to address the effect of each individual factor in performance, when collapsed across all ISI values. 

Spatial factor

We calculated overall performance for each individual subject as a function of the location in the array, grouping the data across all ISI values and letter identities ( Figure 1C). We observed a significant difference in performance across positions, with a maximum at the center of the right side and a population bias for the right vs. the left hemi-field ( p′(right hemi-field): 0.35 ± 0.02; p′(Left Hemi-field): 0.27 ± 0.02; paired t-test: t = 3.7, p < 0.01, df = 18). This spatial population bias in search tasks is well known in the literature and has been related to asymmetries in the spatial allocation of attention, which are related to reading strategies (Efron & Yund, 1996; Goldstein & Babkoff, 2001; Nazir, Ben-Boutayab, Decoppet, Deutsch, & Frost, 2004). Although this bias reflects a trend in the population, the spatial performance map may vary substantially from subject to subject. 

Shape factor

We then analyzed the dependence of performance with form, i.e., whether certain letters were consistently more visible than others ( Figure 1D). We estimated p′ as a function of letter identity—for each individual subject—collapsing the data across all ISIs and spatial locations. The dispersion in performance across different letters was striking, decreasing almost 3-fold from the most visible letter (letter A, 60%) to the less visible letter (letter K, 20%). This result was highly consistent across subjects and could not be accounted by a response bias ( p′ is the performance corrected for false-positives) since the false-positives (the probability of responding to the letter x when it was not presented) were smaller than 3% for all letters. For instance, for 10 of our 19 subjects, the letter A was the most visible letter. The probability that this results from chance (assuming equal probability of responding to each letter for all subjects) is p < 10 ⁻¹⁵. In the next section, we investigate which aspects of letter identity (form, frequency, neighborhood similarity, etc.) account for this observation. 

Letter frequency and letter similarity: The dynamics of confusion

One asymmetry between different letters is their statistics of occurrence, and it is known that letter frequency has an important role in different aspects of perceptual performance (Bramão et al., 2007, Mishkin & Forgays, 1952, Nazir et al., 2004, Ostrosky-Solis, Efron, & Yund, 1991). We investigated the frequency effect by measuring the correlation between performance and letter frequency (Figure 2A). Although there was a consistent trend—all subjects showed a positive correlation—the correlation was very weak and did not reach significance (R² = 0.043, paired t-test, t = 1.58, p = 0.13 df = 18). 

Figure 2

 

Effect of shape and experience in performance and in error distributions. (A) Frequency effect on performance. Average p′—collapsed across ISIs and all positions—as a function of letter frequency. For visualization, each data point is labeled with its corresponding letter. The line indicates the linear regression (points deviating more than 2 standard deviations were considered outliers and shown in red). R² is 0.043 reflecting a very weak correlation. A t-test in which we examined the significance of this correlation (slope different from 0) across subjects did not reach significance (p = 0.13). (B) Confusion matrix. The binary image represents (in orange) the more frequent (p < 0.05) errors as a function of the target letter. The confusion matrix is mainly dominated by morphologic resemblance (see Table 1 for details). (C) Frequency effect on approximate performance. Based on the confusion matrix, we defined “approximately correct responses” when the responded letter was either the target or belonged to its neighborhood of similarity (see text for details). Approximate responses were better correlated with frequency. The slope of the regression (R² = 0.28) was significantly larger than zero (p < 0.01). There is also a significant difference between the regressions of the approximate and the exact responses as a function of frequency (p < 0.05). Outliers are plotted in red. (D) Shape distortion of iconic memory: The entropy of the error distribution increased with ISI, indicating that at short ISIs the error distribution is clustered around the neighborhood of similarity and becomes more heterogeneous for large ISIs. The plot shows that the entropy increase with ISI appears to follow an exponential function with a time constant similar to the one observed in the memory decay.

Another important asymmetry is letter similarity (i.e., how distinct is each letter to the rest of the letters in the alphabet). This is likely to have an impact in performance since in visual search, the visibility of an element decreases as the similarity between target and distractors increases (Duncan & Humphreys, 1989). To explore the effect of letter morphology, we first determined a measure of stimulus similarity by calculating the confusion matrix C. This matrix is calculated first by measuring the number of responses (j) to the stimulus (i) and then estimating the probability that this results from chance under the null hypothesis that all letters are responded with equal probability. Thus, C(i, j) is a measure of the probability that responding to the letter i given that the target was the letter j does not result by chance. All pairs for which C(i, j) < 0.05 are shown in Figure 2B. From an inspection of the most frequent errors (Table 1), it becomes evident that the confusion matrix is mainly dominated by morphologic resemblance; for instance, letter J was mainly confused with letter I, and similarly, the other confused pairs M–W, E–B, W–M, P–R, and I–J correspond to letters with high morphological resemblance. 

Table 1

 

Confusion errors, top 10.

Target letter	Answered letter	% of total errors for the target
J	I	16.0
M	W	12.7
R	S	10.7
E	B	10.4
W	M	10.1
P	R	10.1
N	M	10.1
I	J	9.9
R	A	9.9
S	C	9.7

Based on this, we defined the neighborhood of similarity of a letter as: N( x) =

$\bigcup _{y\ne x}$

y∣ C( x, y) < 0.05, i.e., for each letter x, its neighborhood of similarity N( x) is composed of all letters ( y) such that the probability of responding to y given that the target was x results from chance—is smaller than 0.05. The number of elements in N( x) varies with x between 0 (words that do not have highly probable error targets) and 3. We then estimated a more generalized notion of performance, considering “approximate” responses as those for which the responded letter was either the target or belonged to the neighborhood of similarity. We then estimated the regression between the percent of approximate responses and the frequency of each letter ( Figure 2C) and observed that this correlation is significant ( R ² = 0.28, t-test across subjects p < 0.01, t = 7.3, df = 18). This shows that natural frequency of the letter has an effect on performance, which becomes significant once morphologic contributions to the variance are taken into account. 

This analysis also suggests a critical test for the exponential fading of stimulus information: If shape information is lost exponentially with the fading of iconic memory, then not only performance should decrease; in addition, a trace of this memory should be reflected in the distribution of the errors. The confusion matrix should be more structured at short ISIs—with errors for a letter x being clustered among the closest morphological neighbors. On the contrary, for long ISIs, when shape stimulus information has faded out, we expect errors to be uniformly distributed across the alphabet. To estimate the clustering or homogeneity of the error distribution we measured, for each letter and each ISI value, the entropy of the error distribution:  

$\begin{array}{ccc}{S}_i& =& -\sum _{i\ne j}{p}_{ij}*\ln \left(p_{ij}\right)\hfill \\ \\ p_{ij}& =& T(i,j)*(1-p(i\left)\right)=\frac{T(i,j)}{\sum _{i\ne j}T(i,j)}\hfill \end{array},$

(1)

where S _i corresponds to the entropy of errors for letter i, T( i, j) is the probability of responding the letter j given that the letter i was presented, and p _ij is the normalized probability across errors, i.e., the probability of responding the letter j given that the letter i was presented and the response corresponded to an error trial. The total entropy S is calculated as the mean entropy averaging across all letters. We then calculated the entropy S as a function of ISIs ( Figure 2D) and observed that, as predicted, the entropy of the error distribution increases with ISI. Qualitatively, the time constants of the progression of the entropy distribution and of the decay in performance are comparable. We will later show that this observation can be quantified, performing a second experiment with repeated experimental sessions in single subjects, which allows to determine the parameters of the exponential decay at the individual level. The minimum and the maximum values of the entropy (which is measured in bits) provide a measure of the range of clustering in the error distribution for varying ISIs: In a fully uniformly distributed error distribution among the 25 remaining letters, the entropy would correspond to ln(25) ≈ 3.22; on the contrary, in the case in which all errors would be clustered in only four neighbor letters, the entropy would be ln(4) ≈ 1.39. 

Summarizing, letter identity severely affects the accessibility of a stimulus to working memory. These effects can be explained by the degree of morphological similarity and by an experience-dependent factor determined by letter frequency. Moreover, the clustering of the error distribution decreases with time, suggesting that shape information is progressively degraded in the sensory buffer. In the next section, we address what elements of iconic memory (access or maintenance) are affected by the different experimental factors by studying the interactions between ISI, position, and letter frequency. 

Interactions between letter identity, space, and time

To understand which aspects of iconic memory are affected by letter frequency, we categorized all letters in six frequency groups and analyzed the dependence of p′ with frequency for short and long ISIs. Performance increased with frequency in a non-significantly different manner both for short and long ISIs ( Figure 3A). To quantify this observation, we performed an ANOVA with ISI (short or long) and frequency (the six categories) as main factors and subjects as a random variable. The main effects of frequency and of ISI were significant (ISI: F = 58.67, df = 1, p < 0.01; frequency: F = 3.99, df = 5, p < 0.01) but the interaction was not significant ( F = 1.19, df = 5, p > 0.1). This indicates that the effect of frequency in performance is comparable at short and long ISIs, i.e., that persistency in iconic memory is not determined by letter frequency. In the next experiment, we will provide further evidence for this, showing that the temporal constant of the decay is unaffected by the frequency manipulation. 

Figure 3

 

Effect of frequency letter and space during the course of iconic memory. (A) Frequency effect does not change during the course of iconic memory. Mean performance ( p′) as a function of letter frequency (divided in six categories) for the two shortest (blue) and largest (green) ISI values. The two curves are roughly parallel indicating a comparable effect of frequency for short and long ISI values. An ANOVA analysis assessed quantitatively this observation, indicating that there is a main effect of ISI and frequency ( p < 0.01) but no significant interaction ( p > 0.1). (B) The effect of position changes during the course of iconic memory. Mean performance ( p′) as a function of letter frequency (sorted from higher to lower performance) for the two shortest (blue) and largest (green) ISI values. The effect of position is more pronounced for short ISIs reflecting an interaction between the ISI and the position factors, confirmed by an ANOVA analysis that showed a main effect of space and ISI ( p < 0.01) and a significant interaction ( p < 0.01). Outliers (more than two standard deviations) are indicated in red.

Next, we used the same strategy to understand which aspects of iconic memory are affected by spatial location of the target. We studied the dependence of p′ with spatial location (sorted according to performance when collapsing across all ISI values). The effect of position was considerably more pronounced for short ISI values ( Figure 3B). To quantify this observation, we performed an ANOVA with ISI (short or long) and position as main factors and subjects as a random variable. The main effects of position and of ISI were significant (ISI: F = 70.68, df = 1, p < 0.01; position: F = 141.88, df = 7, p < 0.01). The interaction between ISI and position was significant ( F = 3, df = 7, p < 0.01) in contrast with what we had observed for the interaction between frequency and ISI, which was not significant. This indicates that there are significant differences in the spatial distribution of performance during the few-hundred milliseconds between stimulus presentation and response. 

Estimating the effect of experimental manipulations on the parameters of the exponential decay

The previous results were based on a population of naïve subjects, in which the gain of iconic memory was considerably small: the change in p′ from the shortest to the longest SOA was 0.14, which, with a stimulus display of eight letters, correspond to 1.13 letters. Moreover, of the 24 subjects that participated in the first study, only 13 showed a reliable fit to an exponential function (set to the criterion: R ² > 0.6 and 15 ms < τ < 800 ms). We conducted a second experiment in which we studied performance in multiple sessions of two highly practiced subjects (authors MS and MG) with two aims:

1.  
   to collect enough trials in an individual subject basis to test an exponential model of decay function and understand the effect of the experimental manipulations on the different parameters of the exponential and
2.  
   to assure that we were measuring the limits of iconic memory rather than the quality of iconic memory in inexperienced subjects.

We first analyzed p′ as a function of the delay and, for each individual subject and session, fitted this distribution to an exponential with three free parameters  

$p^{\prime }=\alpha ·e^{(-t/\tau )}+\beta .$

(2)

Each parameter of the exponential provides a measure of a different aspect of the dynamics of sensory memory: α (the gain) indicates the change in performance between short and long delays, i.e., a measure of the information that accesses iconic memory and does not access working memory or explicit reports after a long-delay, β (performance at 8) indicates the performance level for large delays, i.e., the probability that a stimulus accesses working memory, and τ (the time constant) indicates the temporal constant which characterizes the exponential decay in performance, i.e., the duration of iconic memory. 

For both subjects, we observed a consistent decrease in performance with ISI which could be well accounted by an exponential fit ( Figure 4A, Table 2). The fitted parameters of the exponential were not stable through individual experimental sessions (see Supplementary Table 1). Here we cannot conclude whether this relates to intrinsic variability in performance or other non-stationary phenomenon such as learning because we may simply not have enough trials within each experimental session to provide a stable fit. 

Figure 4

 

Main effects of performance and dynamics of confusion of experienced subjects. (A) Exponential fading of iconic memory: Averaged data performance across 6 sessions of two experienced subjects showed a good fit to an exponential function: p′ = α · e ^{(− t/ τ)} + β, α indicates the gain of iconic memory, β the performance level for long delays (access working memory), and τ the temporal constant of the exponential decay. The dispersion of the regression is shown in green. The inset shows the dependence of performance as a function of spatial location. As with the group results, performance was better in the top-right quadrant. (B) Exponential distortion of the quality of iconic memory, indicated by the entropy of the error distribution. The entropy of the error distribution of each subject increased with ISI, indicating that at short ISIs the error distribution is clustered around the neighborhood of similarity and becomes more heterogeneous for large ISIs. The plot shows a fit to the exponential function S = S _max − α × e ^{(− t/ τ)}, with parameters S _max = 1.45 ± 0.01, 1.78 ± 0.01; α = 0.87 ± 0.01, 0.48 ± 0.02; and τ = 132 ± 5, 112 ± 11; for MG and MS, respectively.

Table 2

 

Parameters of the exponential decay of performance (subjects MG and MS).

Condition	α	β	τ (ms)
Subject MG
General	0.341 ± 0.005	0.450 ± 0.003	217 ± 8
Low-frequency letters	0.34 ± 0.01	0.41 ± 0.01 a	189 ± 18
High-frequency letters	0.34 ± 0.01	0.48 ± 0.01 a	271 ± 34
RVF	0.28 ± 0.03 a	0.55 ± 0.01 a	89 ± 30 a
LVF	0.45 ± 0.02 a	0.34 ± 0.01 a	276 ± 28 a
Subject MS
General	0.28 ± 0.01	0.22 ± 0.01	201 ± 26
Low-frequency letters	0.28 ± 0.02	0.16 ± 0.02 a	354 ± 71
High-frequency letters	0.30 ± 0.02	0.25 ± 0.01 a	150 ± 20
RVF	0.40 ± 0.02 a	0.24 ± 0.01	134 ± 16
LVF	0.17 ± 0.02 a	0.20 ± 0.02	269 ± 105

  ^aIndicates a significant difference in a paired t-test ( p < 0.05).

Both subjects also showed an increase in the entropy of the error distribution, which ( Figure 4B) could be well accounted by an exponential fit. To estimate the time constant of this progression, we fitted the data to an exponential function S = S _max − α · e ^{(− t/ τ)} (see legend of Figure 4 for the parameters of the regression for each subject). In both subjects, entropy increased monotonically with ISI, with a temporal constant not significantly different to that obtained for the iconic memory decay (paired t-test across sessions: t = 0.21, p > 0.1, df = 11). 

We then averaged, for each individual subject, the data across different sessions to assess the interactions between experimental manipulations and ISI. Similarly to what we had found in the group result of naïve subject, we observed an effect of frequency and an effect of ISI which showed no interaction (ANOVA, Sub-MS: ISI, F = 85.68 df = 1 p < 0.05; Frequency, F = 3.25 df = 5 p < 0.05; ISI × Frequency, F = 1.29 df = 5 p > 0.1; Sub-MG: ISI, F = 125.64 df = 1 p < 0.05; Frequency, F = 2.47 df = 5 p < 0.05; ISI × Frequency, F = 1.98 df = 5 p > 0.1). When we fitted this data to an exponential, we observed that the only parameter affected in this regression by the conditions was the additive constant of the exponential β (see Table 2). This further suggests that while there is an increase in performance for high frequency letters, this effect does not change in time, i.e., that persistency in iconic memory is not determined by letter frequency. 

Next we examined the interaction between ISI and the spatial location of the target. We observed an effect of position. In both subjects, the bias was comparable to the group results: Performance in the right was better than in the left visual hemifield (paired t-test: Sub-MS: t = 5.5, p < 0.01, df = 5; Sub-MG: t = 6.5, p < 0.01, df = 5; see also inset Figure 4A) and showed an interaction of Position with ISI (ANOVA, Sub-MS: ISI, F = 217.43 df = 1 p < 0.01; Position, F = 62.11 df = 7 p < 0.01; ISI × Position, F = 6.43 df = 7 p < 0.01; Sub-MG: ISI, F = 157.29 df = 1 p < 0.01; Position, F = 100.95 df = 7 p < 0.01; ISI × Position, F = 7.18 df = 7 p < 0.01), although in one of the two subjects, this interaction is reversed (i.e., spatial changes in performance are shorter for the shorter ISIs, see figure in 1). The analysis of the parameters of the exponential also yielded a more complicated and variable picture of the position manipulation, summarized in Table 2. Thus, while the effect of frequency is to a large extent insensitive to time, the effect of position shows a strong interaction, although the specific pattern of this interaction may vary across different subjects. Such variability may be related to different strategies in the spatio-temporal allocation of attention, and more specific experiments are required to determine a precise model of the evolution of iconic memory in different locations of the visual field. Here we merely provide in 1 a tentative explanation of the difference across subjects. 

Discussion

We investigated whether stimulus location, shape similarity, and shape familiarity—factors which are known to affect overall performance in a search task—target different stages in the dynamics of information processing. We showed that letter frequency affects the access to iconic memory but does not affect the maintenance and selection of information. On the other hand, we observed that the effect of spatial position changed during the course of iconic memory. The precise nature of this change varied across subjects, suggesting different individual patterns of spatio-temporal allocation of attention in the visual scene. Finally, we could track the distribution of errors showing that shape information decays progressively. At short-time (less than 300 ms) after stimulus presentation, errors involve responses which are morphologically similar to the target letter. After a delay of a second; errors were more uniformly sampled across the alphabet. This suggests that iconic memory is distorted progressively in shape space. 

Representing shapes in time

Many lines of evidence from psychophysics (Duncan & Humphrey, 1989; Treisman & Gelade, 1980), single-cell studies in monkeys (Lee, Yang, Romero, & Mumford, 2002; Li, Piëch, & Gilbert, 2006), and human imaging (Altmann, Deubelius, & Kourtzi, 2004) and modeling (Itti & Koch, 2001) have shown that stimulus saliency can strongly modulate perceptual responses. On the other hand, it has been shown that in a cluttered field, the saliency of a stimulus is determined by a generalized distance, in feature space, to the field of distractors (Ashby & Lee, 1991; Duncan & Humphrey, 1989; Feldman, 2003; Feldman & Singh, 2005). This general finding of visual perception has been widely studied in the specific domain of letter perception—and from there on to word recognition (Nazir et al., 2004; Nazir, Jacobs, & O'Regan, 1998)—demonstrating that the matrix of confusion can be determined, to a large extent, by morphological proximity (either by space overlapping or by feature proximity) (Blommaert, 1988; Bouma, 1970, 1971; Chialvo, 1997; Gervais, Harvey, & Roberts, 1984; Townsend, 1971). In the more general case, however, other non-low-level sensory factors as well as acoustic confusions (Baddeley, 1966, 1968) may contribute to letter confusion depending on the specifics of the task (Townsend, 1971). While the aim of this work was not to provide a quantitative derivation of the confusion matrix from a feature metric in letter space, in this specific experiment it was evident from informal inspection that the confusion matrix is largely determined by morphological factors. Our aim was to understand the dynamics of confusion in the course of iconic memory. Subjects' performance decreased exponentially as the ISI increased, as has been repeatedly found in partial report paradigm studies (Sperling, 1960). The comparisons between a large population of naïve subjects and highly practiced subjects indicated that trained subjects generate reliable exponential decaying performance and thus that this decay reflects an intrinsic limit of iconic memory, while the variable responses in naïve subject may be related to the amount and quality of iconic memory that can demonstrated with relatively little practice (Sperling, 1960). Further, we observed a decrease in the error distribution clustering in a comparable time scale, evolving from errors that were clustered in close neighbors of the target letter to a more uniform distribution. This result was very robust and was observed in every individual session of the two experienced subjects. Such temporal drift in the error distribution argues for a progressive distortion in shape space and a non-categorical representation storage of iconic memory, as has been suggested by other studies (Turvey & Kravetz, 1970; Von Wright, 1970). 

The effect of experience in the dynamics of perception

We used in our experiment letter frequency as an indicator of experience related modifications of sensory processing since many studies have demonstrated that as a result of exposure to print, neural detectors become dedicated to the recognition of frequent fragments that are useful to encode existing words (Ben-Shachar, Dougherty, Deutsch, & Wandell, 2007; Binder, Medler, Westbury, Liebenthal, & Buchanan, 2006; Dehaene, Cohen, Sigman, & Vinckier, 2005; Pammer et al., 2004; Price, Wise, & Frackowiak, 1996; Vinckier et al., 2007). Beyond reading and word recognition, adult perceptual learning plays a major role in shaping perception (Gilbert, Sigman, & Crist, 2001). In particular, in search tasks, it has been shown that learning can endow a specific shape with pop-out characteristics (Li, Piëch, & Gilbert, 2008; Lee et al., 2002; Sigman & Gilbert, 2000; Sigman et al., 2005; Wang, Cavanagh, & Green, 1994). Yet, there is not a clear understanding on which aspects of processing are changed during the course of perceptual learning that may account for the observed pop-out effect. Single-cell studies in awake behaving monkeys have shown that perceptual learning can affect neuronal responses from the first spikes (Crist, Li, & Gilbert, 2001; Li, Piëch, & Gilbert, 2004), leading to the prediction that learning may modulate the access of a stimulus to iconic memory through an early amplification of the first wave of sensory responses. On the other hand, there was no clear prediction on whether frequency would affect or not maintenance in iconic memory. Our results showed that letter frequency does not affect significantly the duration of iconic memory. This result may explain the difficulty of holistic word recognition (Pelli, Farell, & Moore, 2003) since features are simply not available for sufficient time to be bound as a whole, and the maintenance of a stimulus in sensory buffer may be moderately susceptible to experience. 

Spatio-temporal sampling of the visual scene

In our data, we observed like other authors (Efron & Yund, 1996; Gegenfurtner & Sperling, 1993; Latimer, Stevens, Irish, & Webber, 2000) a superiority effect on the right visual field as compared to the left visual field, as well as a superiority of the upper visual field when compared to the lower visual field. It has been argued that the right to left asymmetry may result from a reading bias (attending to the right of the fovea) (Bramão et al., 2007; Latimer et al., 2000; Mishkin & Forgays, 1952; Nazir et al., 2004; Ostrosky-Solis et al., 1991). The superiority of the upper visual field respect to the inferior one has also been reported in many studies (Goldstein & Babkoff, 2001; Mishkin & Forgays, 1952; Previc, 1990), and it has been proposed that it may be related to an asymmetry in natural vision: objects closer to the observer appear lower in the visual field, and distant objects are seen above the horizon. Thus the lower field may be comparatively more specialized in global and low-spatial frequency process and the upper hemifield in high frequency visual search which may be required for far vision. As mentioned previously, this is the result of a population bias; the specific map of performance as a function of spatial location may vary substantially from subject to subject. As with our other analyses, we concentrated in understanding which aspects in the dynamic of sensory processing may result in this asymmetry. In the group analysis, we observed that spatial asymmetries are more prominent during short intervals between stimulus presentation and response and vanish as this time increases, indicating that the gain of iconic memory is affected by spatial position. When we investigated this effect in experienced subjects, we observed that this interaction could be reversed. A detailed analysis of the dynamics of performance for different spatial locations (1) showed that the dynamic range of iconic memory (where we observe a more substantial change between short and long ISI values) is observed at intermediate performance levels. The reversal thus corresponds to the results of a subject in which performance of the right field is close to saturation. Hence, the decay with ISI is more pronounced in the left hemifield where the levels of performance are closer to the higher dynamic range. 

While this data cannot conclusively determine the mechanism underlying these observations, a speculative explanation may involve a non-homogeneous temporal distribution of top-down control of the visual stimulus. In the simplest hypothesis which may account for the data of the group result, it is possible that task-expectation mechanisms may address top-down control in a specific portion of the visual field (right and upper hemi-fields), thus augmenting the probability that stimuli in this region access iconic memory and increasing the gain as observed in our experiments. Alternatively, this may also involve a sequential sampling of the visual scene; the first scanned locations would show an initial bias, but as the whole scene is scanned this difference should vanish. An internal scanning mechanism has been proposed by Efron, Yund, and Nichols (1987) for a visual search task and other authors (Latimer et al., 2000), although possible alternatives have been also proposed (Efron & Yund, 1996; Efron, Yund, & Nichols, 1990; Wolfe, 1998) and this result remains controversial. Also, in a study in which we investigated performance in a search task during shape training, we showed that learning could be position specific even when there were no spatial cues in the task (the target could appear anywhere). Retinotopic specificity in this task resulted therefore from intrinsic mechanisms that we proposed could reflect the sequence of sites targeted by the search strategy (Sigman & Gilbert, 2000). Rapid sequential shifts of attention have been identified in (Woodman & Luck, 1999) using an electrophysiological marker of the moment-by-moment direction of attention, although in this experiment, search order was biased controlling the probabilities of target occurrence. It is thus likely that in natural vision scanning order may be considerably less deterministic yet biased by consistent strategies developed during visual experience. Finally, the results on the highly practiced subjects suggest that almost perfect performance for some locations, independently of ISI, can be achieved. Interestingly, this was observed in the subject that had more substantial practice (MG), suggesting that sufficient practice may lead to a high-capacity long-lasting memory of the iconic buffer. 

Materials and methods

Subjects

Twenty-five native Spanish speakers (12 male, 13 female) with an age range of 19–35 participated in this experiment. All the subjects reported normal or corrected-to-normal vision. They were all graduate and undergraduate students from Faculty of Exacts and Natural Science, University of Buenos Aires (Buenos Aires, Argentina). All subjects gave written consent to participate in this study. Twenty-four subjects participated in the first experiment; two subjects (authors MS and MG) participated in experiment two. MG also participated in the first experiment. 

Visual stimuli and procedure

Visual Stimuli were presented on a PC. Behavioral experiments were programmed in Python ( http://www.python.org). In each trial, eight letters were presented simultaneously for 106 ms (corresponding to 9 frames with a refresh rate of 85 Hz) on the screen after 1000 ms of fixation. The stimuli were presented on a 19-in. screen (resolution of 800 × 600 pixels) placed at a distance of 73 cm in front of the subject. The letters were created using the Time New Roman font. Each letter was chosen randomly from the alphabet (26 symbols) and showed in uppercase, with a size of 1.2°. The eight letters were arranged on a circle, around the fixation point at an eccentricity of 5.2°. A red circle (0.1 °) on an array of blue ones (with the same configuration of the letters) indicated the target position. Subjects were asked to report the letter indicated by the red circle (with the keyboard). Eight inter-stimulus intervals (ISI) were used (24, 71, 129, 200, 306, 506, 753, and 1000 ms). In all conditions, the cued stayed on until the response. 

Experiment 1

Each observer completed an entire practice block of 64 trials before the main experiment. Subsequently, subjects completed 6 blocks of 64 trials (total, 384 trials). In each block, all positions (total 8) and all ISIs (total of 8) were randomly and uniformly sampled. Subjects were instructed to fixate in the center of the screen during the entire experiment and to report the letter as fast as they could. The session lasted <40 min. 

Experiment 2

Two subjects with previous experience in psychophysics and in partial report paradigms (authors MS and MG) conducted six experimental sessions. Each subject completed 6 sessions of 6 blocks each one (384 trials for session) identical to the used in Experiment 1. All sessions were performed in different days. Subjects were instructed to fixate in the center of the screen during the entire session and to report the letter as fast as they could. Each session lasted approximately 30 min. 

Data analysis

For the first study, we examined 24 subjects. Nineteen of these 24 showed a consistent decay of performance with ISI, but the other five produced unreliable data in which we could not measure this main effect, indicating that this is a demanding task that requires a certain amount of practice to achieve stable performance. According to this, we only consider for further analysis the 19 subjects satisfying the decay of performance with time. 

Data were corrected by false-positives (FP), using the following equation  

$p^{\prime }=\frac{\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{o}\mathrm{f}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{e}\mathrm{s}-\mathrm{F}\mathrm{P}}{1-\mathrm{F}\mathrm{P}}.$

(3)

FP were defined as the response probability for a specific letter given that it was not presented as the target. FP were calculated for each individual letter, independent of the ISI value. FP were bellow 3% for all conditions, and thus corrected performance was not substantially different than the non-corrected performance. 

In Experiment 2, we conducted a longitudinal experiment in which we measured performance for subjects (authors MS and MG) during repeated sessions, and the time-course performance (as a function of ISI) was fitted to an exponential function with parameters A, B, and τ  

$p^{\prime }=\alpha ·e^{(-t/\tau )}+\beta .$

(4)

Individual data sessions and averaged data were fitted to this exponential function. 

Correlations using frequency letters

Letter frequencies in Spanish were obtained through the analysis of the 80,000 more frequent words by the Spanish corpus LEXESP (Sebastian-Gallés, Marti, Carreiras, & Cuetos, 2000). 

Statistics and regression analysis

Statistics were done through t-test student comparisons and ANOVA, assuming a normal distribution for the data. For linear regression analysis, we discard the data points bigger than two standard deviations (but they were shown in the figures in red). 

Supplementary Materials

Supplementary Table 1 - Supplementary Table 1 

Supplementary Table 1. Mean parameters for each individual session, Subject MG and MS. 

Supplementary Figure 1 - Supplementary Figure 1 

Supplementary Figure 1. Variability in the interaction between space and time across different subjects. The change in performance as a function of spatial position was different in the right (blue) and left (green) visual fields, indicating an interaction between space and ISI. This interaction was distinct in two different subjects. To address wether these differences could be accounted by saturation effects of the iconic memory decay (see Appendix A) we plotted (inset of B) p′ for positions of high, medium and low performance. In both subjects, the maximal decrease, indepedently of the spatial location, was observed at locations of intermediate performances. 

Appendix A

Subject variability in the interactions between space and time

In the Results section, we showed that the interaction between spatial location of the target and ISI showed different patterns in the two experienced subjects ( Supplementary Figure 1). To understand what may determine the specifics of this interaction, we examined, for each subject, the dynamics of performance for spatial locations as a function of performance. For each subject, we binned the data in three categories: locations of high performance

$\{x:p^{\prime }(x)>\frac{2}{3}·{\max }_x(p^{\prime }\left(x\right)\left)\right\}$

, locations of average performance

$\{x:\frac{2}{3}>p^{\prime }(x)>\frac{1}{3}·{\max }_x(p^{\prime }\left(x\right)\left)\right\}$

, and locations of low performance

$\{x:\frac{1}{3}>p^{\prime }(x)·{\max }_x(p^{\prime }\left(x\right)\left)\right\}$

and plotted for each of these regions the performance as a function of ISI ( Supplementary Figure 1, inset). This analysis indicated that, despite the specific spatial specificity, the iconic memory decay was more significant for locations in which performance at the shortest ISI was intermediate. Average performance in some locations was very close to perfect visibility ( p′ ∼ 1) and thus close to saturation and with very low variations of performance with ISI. In other locations, performance was very low ( p′ ∼ 0.2). In these locations, we also observed a very modest improvement for short ISIs. On the contrary, in locations in which average performance was intermediate ( p′ ∼ 0.5), the fraction of correct responses showed the most significant change with ISI. This result can explain the reversal of the interaction between the two subjects. For most subjects (of the naïve group and subject MS), the maximum of performance across spatial locations (right visual field) corresponded to intermediate levels of performance that are strongly affected by ISI, and the minimum of performance corresponded to very poor levels of performance thus showing a more modest effect of ISI. Subject MG, who had large amounts of practice in this task, showed saturation for the most performing locations and intermediate levels of performance for the worse locations, hence the inverted effect. It might be interesting to consider, in further studies, whether achieving close to perfect levels of performance in partial report paradigms might be achieved with sufficient extensive training. 

Acknowledgments

We thank George Sperling for useful comments and suggestions. MS is funded by the Human Frontiers Science Program and by the SECYT, PICT 38366. MG has a fellowship of the National Research Council of Argentina (CONICET). 

Commercial relationships: none. 

Corresponding author: Mariano Sigman. 

Email: [e-mail]. 

Address: Integrative Neuroscience Laboratory, Physics Department, Universidad de Buenos Aires, Argentina.