Object representations in visual memory: Evidence from visual illusions

Abstract

Abstract

Abstract:

Abstract  Human visual memory is considered to contain different levels of object representations. Representations in visual working memory (VWM) are thought to contain relatively elaborated information about object structure. Conversely, representations in iconic memory are thought to be more perceptual in nature. In four experiments, we tested the effects of two different categories of visual illusions on representations in VWM and in iconic memory. Unlike VWM that was affected by both types of illusions, iconic memory was immune to the effects of within-object contextual illusions and was affected only by illusions driven by between-objects contextual properties. These results show that iconic and visual working memory contain dissociable representations of object shape. These findings suggest that the global properties of the visual scene are processed prior to the processing of specific elements.

Introduction

The processes underlying the neural transformation of the visual input into conscious experience have been widely considered along concepts of informational processing. Several theories have been put forward to describe how such computations take place in the visual brain. Some argued that the computation is accomplished in a feed-forward manner. Object recognition, for example, should be a hierarchal bottom-up buildup of information; simple feature extraction, starting in early visual areas (contours, figure ground segregation), which in turn gives rise to the creation of more complex featural representations and eventually leading to an inclusive object representation (Marr, 1982; Biederman, 1985; Riesenhuber & Poggio, 2000). Others argue that perception can be described as a mixture of top-down and bottom-up processes in which context, experience, and expectations play a central role in the formation of object representations. More recent functional and behavioral studies have supported such a model of perception using different experimental methods (Bar et al., 2006; Hochstein & Ahissar, 2002) by showing that top-down processes are an integral part of online perception, and that some of the information entering early visual areas is sent forward to frontal areas in parallel to the feed-forward processing along progressive cortical layers. In the current study, we tested whether early representations of object shape and object size along the visual memory hierarchy are modulated by top-down processing. 

Visual working memory (VWM) is considered by many to contain relatively elaborated representations along the processing hierarchy of visual information (Todd & Marois, 2004; Vogel, Woodmen, & Luck, 2001). Visual information stored in VWM is considered to be the result of earlier automatic perceptual processes, which may also be affected by top-down information (Patterson & Martin-Bly, 2007). Iconic memory is considered to be located earlier along the visual processing hierarchy, and it has been shown that certain strategies can be used to consciously access that information (Averbach & Coriell, 1961; Coltheart, 1980; DiLollo & Dixon, 1988; Dixon & Gordon, 1997; Sperling, 1960; Sligte & Lamme, 2008). It is less clear, however, whether iconic memory is or is not affected by top-down information. Recent imaging studies provide evidence that visual illusions of size (e.g., the Ponzo illusion and the Ebbinghaus illusion) affect neural activation in early visual areas, including V1 (Fang, Boyaci, Kersten, & Murray, 2008; Murray, Boyaci, & Kersten, 2006). These results suggest that iconic memory should be affected by this class of visual illusions. Note that the Ebbinghaus and the Ponzo illusions are based on relatively complex relational properties between the target object and its surrounding environment, which comprises the visual illusion (for example, see Figure 1). We will term this class of illusions as illusions that stem from between-objects relational properties. Based on previous imaging findings, we hypothesize that iconic memory would be affected by this class of visual illusions. In addition, we will test whether or not iconic memory is affected by a different class of visual illusions, those relying on within-object relational properties (Ganel & Goodale, 2003, see Figure 1). Note that within-object illusions were rarely used in the literature and, as such, the relationship between within-object and between-objects illusions is presently unexplored. 

Figure 1

 

Examples of target stimuli used in Experiment 1 (left panel). In this version of the Ebbinghaus illusion, a circle surrounded by bigger circles is perceived smaller than an identical circle surrounded by smaller circles. The number of surrounding circles was kept to three to avoid intervening numerical effects. The target stimuli used in Experiment 2 (right panel). Although they are of equal width, the shorter rectangle is perceived wider than the taller rectangle (see Ganel & Goodale, 2003).

We used the classic psychophysical method of constant stimuli to compare the nature of representations stored in VWM and in iconic memory. The main independent variable was the contextual effects of visual illusions. Visual illusions provide a powerful tool for probing contextual effects and are generally thought to be cognitively sealed (Gregory, 2009). For example, in the Ebbinghaus illusion, a circle is perceived as either larger or smaller than its actual size as a function of the size of the surrounding circles. Interestingly, there is evidence that visual illusions such as the Ebbinghaus illusion do not affect the online control of grasping (Haffeanden & Goodale, 1998; but see Franz, Gegenfurtner, Bülthoff, & Fahle, 2000). It is possible, therefore, that accurate information about object size can be still available for the visual system at early stages of visual processing, despite the fact that a subject's conscious perception is affected by the illusion. Note that it has been shown that memory-based visuomotor control can be affected by visual illusions (e.g., Goodale, Westwood, & Milner, 2004). By unconsciously accessing the raw information of object metric properties, the motor system can interact with greater accuracy with the real world but, of course, only to the extent that such information is actually available. It could be the case, therefore, that object representations in VWM and in iconic memory would be differentially affected by veridical and illusory information. For example, object representations in iconic memory may contain more veridical information compared to VWM. A second and even more intriguing possibility is that object processing is affected by global (or holistic) contextual information even at initial processing stages (Hochstein & Ahissar, 2002), which in turn would lead to similar effects of visual illusions on VWM and on iconic memory. 

The idea that conscious perception contains only a fraction of the information stored in iconic memory is highlighted in Sperling's classical partial report experiments (Sperling, 1960). This study provided strong evidence that the information stored in early visual memory is much larger in scale than participants are able to consciously report. The iconic memory storage, as was later termed by Neisser (1967), was thought to be a sort of rapidly decaying sensory memory with almost unlimited capacity. A more recent model of iconic memory proposed by Coltheart and his colleagues (Coltheart, 1980; see also DiLollo, 1977; Efron, 1970; Eriksen & Collins, 1967; Irwin & Brown, 1987; Irwin & Yeomans, 1986; Loftus & Irwin, 1998) argued that iconic memory can be divided into three serial processing stages: neural persistence, visual persistence, and informational persistence. Neural persistence represents the neural activation elicited by a certain input following its removal, from the firing early retinal receptors up to the activation in early visual areas. Visual persistence is the time window in which the perceiver still perceives the stimuli although it not actually presented. Informational persistence, according to this model, is what has been probed by Sperling's classical partial report paradigm. Most studies that have used this paradigm have used relatively complex objects (i.e., digits, letters, and words) for brief presentations (Averbach & Coriell, 1961; Merikle, 1980; Sperling, 1960). More recent studies (i.e., Becker, Pashler, & Anstis, 2000; Sligte & Lamme, 2008) have used the partial report paradigm with simpler stimuli, such as two-dimensional geometrical shapes. These studies have also shown that, when a spatial cue is given to a specific object location before its icon decays, various visual features of the object (its orientation, color, or shape) can be readily extracted. 

It has been argued that the differences between iconic memory and VWM stores reflect our phenomenal experience of the visual environment (Block, 2007). This is illustrated by the difference between our seemingly rich phenomenal experiences of the visual scene contrasted with our sparse ability to report such phenomenal knowledge. Block argued that the evidence for different representations in iconic memory suggest that our phenomenal consciousness is much richer than our ability to cognitively access that experience (access-consciousness). Indeed, if iconic memory corresponds to our phenomenal experience of the world, it would be reasonable to expect that visual illusions would affect representations in that storage. 

Here we used different time-windows to probe the nature of object representations during initial and advanced stages of visual processing. It is assumed that iconic memory and VWM are distinct memory storages (Awh, Barton, & Vogel, 2007; Lamme, 2004; Patterson & Martin-Bly, 2007; Vogel et al., 2001) and that iconic memory is located earlier along the visual hierarchy (Becker et al., 2000; Coltheart, 1980; DiLollo, 1977; Irwin & Yeomans, 1986; Lamme, 2010; Long, 1980; Sperling, 1960). Yet, it is not clear whether the flow of information between the two systems is achieved in a parallel manner, or alternatively, whether it is achieved via a feed-forward or bottom-up manner. Based on earlier imaging findings (Fang et al., 2008; Murray, Boyaci, & Kersten, 2006), we predict that object representations in iconic memory should be affected by (top-down) contextual information and could, therefore, be affected by visual illusion. Conversely, if objects are processed in a bottom-up manner, it is expected that iconic memory would show greater sensitivity to the metric properties of objects (and hence, would show smaller or no effects of illusions) compared to VWM. 

Experiment 1

The experimental procedures and timing were based on previous iconic memory (see Coltheart, 1980; Irwin & Yeomans, 1986) and change blindness (Becker et al., 2000; Sligte & Lamme, 2008) studies. In Experiment 1, we used the Ebbinghaus illusion to compare the nature of representations stored in VWM and iconic memory. Participants were presented with different versions of the illusion for a brief duration of time in each trial. A mask appeared soon after the target disappeared, and participants were asked to indicate if a probe (that contained only the middle circle of the illusion presented in isolation) was bigger or smaller than the target circle. We expected VWM representations to be affected by contextual information. Would iconic memory, compared to VWM, be differently affected by the illusion? 

Methods

Participants

Twelve undergraduate students received course credit or were paid for their participation in the experiment. All had normal or corrected-to-normal vision. Participants were randomly assigned to the iconic or the VWM experimental groups. All ethics were approved by the local ethics committee. 

Stimuli

Stimuli were presented on a computer screen. One of two target illusions was presented on a black background. The central circle diameter was constant (30 mm) while the surrounding circles could be either 50 mm or 15 mm in diameter. The probe stimulus was one of six different circles (27, 28, 29, 31, 32, and 33 mm in diameter). The probe circle was presented 10 mm below the original target. The noise mask was a 50 × 50 mm stimuli constructed by assigning random brightness values to each pixel using Adobe Photoshop software (uniform noise monochromatic filter at 400% strength). 

Procedure

Participants received general instructions and were seated about 45 cm away from the LCD screen. Each trial began with a fixation point presented for 1000 ms followed by the target stimulus which was presented for 80 ms. On half of the trials, the target was surrounded by large circles (50 mm) and on the other half, by small circles (15 mm). Participants were asked to refer to the central circle and to compare it with the probe. A blank interstimulus interval (ISI) was then introduced for a duration of 200 ms followed by a 60-ms noise mask. In the iconic memory group, participants were presented at this stage with the probe stimulus, which remained on the screen until a response was made. The same procedure was used for the VWM group with the exception that a 1500-ms ISI was introduced prior to the presentation of the probe. Trials were separated by 3000-ms ISIs. Each of the probe circles (out of possible six) was repeated 15 times in combination with each target (a total of 180 randomized trials throughout the experiment). Participants were asked to make accurate and fast size classifications comparing the probe circle with the central circle of the illusion by pressing the “K” keyboard key to indicate that the probe was larger than the target or “F” to indicate it was smaller. Participants were told that the probe was always either larger or smaller than the target and that they should make a guess if they were not sure about whether the probe stimuli was larger or smaller than the target. Eight practice trials, for which feedback was given, were administrated prior to the presentation of the experimental trials. 

Results and discussion

We counted the number of times each subject replied “larger” for each of the probes in each condition. The dependent variable was extracted by fitting a logistic function to the data and computing the size value at the point of chance accuracy (50% correct). This PSE (point of subjective equality) is the point where the subject is not able to determine whether the probe is larger or smaller than the target and represents its perceived size. The goodness of fit of each subject to a logistic function was measured ( $mean{r}^2=.98$; $std\hspace{0.5em}{r}^2\hspace{0.5em}\pm \hspace{0.5em}.02$). As can be seen in Figure 2, both iconic memory and VWM were modulated by the Ebbinghaus illusion. The data were submitted to an analysis of variance (ANOVA) with memory store (iconic, visual working memory) as a between-subject variable and illusory context (large circles, small circles) as a within-subject variable. A significant main effect of the illusion was found (F[1, 10] = 12.834; MSE = 0.66; partial eta square = 0.562; p < 0.01) with no interaction between memory type and illusion context (F[1, 10] = 0.02), indicating the target object being perceived as smaller when surrounded with large compared to small circles in both types of memory representations. This main effect indicates that iconic memory represents visual information in a similar way to the way this information is represented in VWM. 

Figure 2

 

Effects of the Ebbinghaus illusion on iconic memory and on VWM in Experiment 1.

Note that this is more likely to be accommodated by the top-down rather than the bottom-up view of perceptual hierarchy. The illusory effects of the Ebbinghaus illusion are based on contextual effects between different objects. That is, the perceived size of the target circle is computed relative to the sizes of the surrounding circles and cannot be perceived in an isolated, veridical manner. Thus, iconic memory might contain certain aspects of holistic representations, including a relative perception of size. 

Other studies have extended the idea of contextual processing to situations in which the target stimulus is defined by a single feature of the object while context is defined as other, task-irrelevant parts of the same object. For example, Ganel and Goodale (2003) showed that the perception of one dimension of an object cannot be achieved independently from other dimensions of the same object (also see Felfoldy, 1974). In particular, perceptual judgments of a rectangle's width were found in this study to be contingent on its length with long rectangles perceived as narrower compared to short rectangles (Ganel & Goodale, 2003). These findings illustrate a Gestalt processing of object shape in which single parts belonging to the same object cannot be represented in an isolated manner and are rather perceived in relative terms. Furthermore, these findings also show that the contextual effects observed in visual illusions within different objects can be also found in situations in which an irrelevant object's dimensions create an illusory percept of a single object feature. 

Experiment 2

The purpose of the following experiment was to test if iconic memory and VWM representations are based on a similar Gestalt processing of object shape. In other words, we tested whether the relative processing found among different objects in Experiment 1 would extend to relative processing among different object dimensions. To this end, we used a modified version of the paradigm used by Ganel and Goodale (2003) to compare object representations stored in iconic memory and in VWM. 

In this experiment, we tested whether elements of simple objects (rectangles) are represented in an isolated or in a Gestalt holistic manner. To this purpose, we adapted a methodology previously used by Ganel and Goodale to test if the perception of a rectangle's width can be achieved independently of its length. Participants were presented with a target of a single 2D rectangle for a brief duration followed by blank and noise masks. Following a varying ISI, participants were presented with a single line probe. Their task was to make size discriminations between the length of the probe and the length of the line embedded within the target object (e.g., the rectangle's width). We manipulated the irrelevant dimension of the rectangle's height and tested its effect on width perception. Ganel and Goodale (2003) showed that perceptual estimations of width are affected by height with taller rectangles perceived as narrower compared to shorter rectangles (Figure 1). Such a distortion in the perception of a single element is the result of holistic or relative processing of object shape. 

Methods

Participants

Twenty undergraduate students received course credit or were paid for their participation in the experiment (eight in the VWM group, 12 in the iconic memory group). All had normal or corrected-to-normal vision. Ethics were approved by the local ethics committee. 

Stimuli

Stimuli were presented on a computer screen. One of two target rectangles was presented on a black background. The rectangle's width was constant (30 mm) and the length was either 30 or 50 mm. The probe stimulus was one of six different lines (27, 28, 29, 31, 32, or 33 mm). The probe line was presented 10 mm below the target rectangle. 

Procedure

The procedure was similar to the one used in Experiment 1, but now, on half of the trails, the target was a 50 × 30 mm rectangle and, on the other half, a 30 × 30 mm rectangle. Participants were asked to refer to the rectangle's width (horizontal lower side of the rectangle) and to later compare this dimension with the line probe. 

Results and discussion

The data was analyzed in a similar manner to the analysis used in Experiment 1. The results are presented in Figure 3. The goodness of fit of each subject to a logistic function was $r^2=$ 0.97 $\hspace{0.5em}(std\hspace{0.5em}{r}^2\pm \hspace{0.5em}.04$). As seen in the figure, the illusion affected VWM with longer objects perceived as narrower than shorter ones. However, unlike in Experiment 1, the illusion did not affect iconic memory. The data were submitted to an analysis of variance (ANOVA) with memory store (iconic, visual working memory) as a between-subject variable and illusory context (short or long rectangle) as a within-subject variable. A significant interaction was found between the illusion and memory store (F[1, 18] = 6.78; MSE = 0.37; partial eta square = 0.273; p < 0.05). Further analysis revealed that the interaction resulted from a simple main effect of length in the VWM group (F[1, 18] = 8.25; MSE = 0.29; p < 0.025) but not in the iconic memory group (F[1, 18] = 0.357). 

Figure 3

 

Effects of the rectangle's irrelevant height on width perception in Experiment 2.

These findings show that, unlike the iconic memory storage that was affected only by the Ebbinghaus illusion, the VWM was affected both by the Ebbinghaus illusion and the rectangle's irrelevant height. These findings support the notion that iconic memory and VWM contain distinct representations, and that the paradigm used in our experiment is sensitive to revealing potential difference between the two memory stores. More interestingly, these results provide a first hint that certain aspects of holistic processing in VWM do not characterize iconic memory. Indeed, while the results of Experiment 1 provide evidence favoring top-down representations in iconic memory, the results of Experiment 2 suggest that such effects could be limited to between-objects relationships but not to within-object dimensional relationships. 

Thus, illusions based on relational properties between objects seem to affect representations in iconic memory, whereas illusions based on relational properties within an object seem to have no effects on representation within the iconic memory store. However, before embracing this idea, it is important to test its generality using converging evidence from other types of visual illusions. If our hypothesis is correct, we would expect to find that other illusions originating from between-objects contextual effects would affect both iconic memory and VWM. However, illusions that originate from within-object dimensions are expected to affect only VWM but not iconic memory representations. Our next experiment was designed to further test this hypothesis using a different set of visual illusions. Unlike the illusions used in Experiments 1 and 2, which are modulated by relative processing of size, the two illusions used in Experiment 3 are triggered by pictorial depth cues. 

Experiment 3

In Experiment 3, we used two types of visual illusions based on pictorial depth cues which are known to distort veridical size perception. A version of the Ponzo illusion was used as the between-objects illusion, and a version of the 3D cube illusion (see Maniatis, 2010) was used as the within-object illusion (see Figure 4). We chose these illusions for several reasons: First, both illusions are based on the same underlying principle but differ in the locus of the illusory effect (e.g., whether the illusions originate from between or within-object properties). Therefore, the results of Experiment 3 could provide converging evidence to the findings of Experiments 1 and 2 using a different controlled set of illusions. In addition, the displays used in Experiment 3 also answer a potential issue in the design used in Experiments 1 and 2. In particular, it could be argued that the different pattern of results observed in our earlier experiments was triggered by shape differences of probes used in the two experiments (circles in Experiment 1 versus lines in Experiment 2). In Experiment 3, both between-objects and within-object illusions were probed using an elongated line. We expected to find a similar pattern of results to the ones found in Experiments 1 and 2. In particular, the Ponzo illusion was expected to affect both VWM and iconic memory, whereas the 3D cube illusion was expected to affect VWM but not iconic memory. 

Figure 4

 

Left panel: The Ponzo illusion used in Experiment 3. Only one of the lines was shown in each exposure and participants were asked to compare each target line with a line probe. Right panel: The three-dimensional cubes. Note that although the lines are of equal size, the lower horizontal line is perceived smaller in the right compared to the left cube.

Methods

Participants

Twenty-seven undergraduate students received course credit or were paid for their participation in the experiment (13 in the VWM group: six in the Ponzo condition and seven in the 3D-cube condition, and 14 in the iconic memory group: seven in each condition). All had normal or corrected-to-normal vision. Ethics were approved by the local ethics committee. 

Stimuli

In the Ponzo illusion condition, targets were made of two converging black lines with a single horizontal line embedded between them, which was lower (closer in depth) or higher (farther in depth) in the visual field. Targets were placed on a light-gray background. In the 3D-cube illusion condition, the target was a white, filled, three-dimensional cube with black contour lines that were 30 mm in length and width. The cube depth dimension was either 20 mm or 50 mm. The probe stimulus was one of six different lines (27, 28, 29, 31, 32, or 33 mm) and was presented 10 mm below the target. 

Procedure

The procedure was similar to the one used in Experiment 1. On half of the trails, the target was the perceptually smaller and, on the other half, perceptually larger. Participants were asked to refer to the target width (for the 3D cube, the horizontal upper side of the cube) and to later compare this dimension with the probe. 

Results and discussion

The results were analyzed as in the previous experiments and are presented in Figure 5. The goodness of fit of each subject to a logistic function was r² = .97 (std r² ± .02). As can be seen in the figure, both iconic memory and VWM yielded similar Ponzo illusion effects (see Figure 5b). In contrast, for the 3D-cube stimuli, only VWM produced illusory effects with no illusory effects in the iconic memory condition (see Figure 5a). The data were submitted into an analysis of variance (ANOVA) with memory store (iconic, visual working memory) and illusion type (Ponzo, 3D cube) as between-participants variables and the illusion direction (perceptually smaller, perceptually larger) as a within-subject variable. A significant interaction was found between the illusion type, memory store, and illusion direction (F[1, 24] = 7.21; MSE = 0.41; partial eta squared = 0.231; p < 0.05). This three-way interaction resulted from a significant two-way interaction between memory store and illusion direction in the 3D-cube group (F[1, 24] = 9.95; MSE = 0.22; partial eta squared = 0.433; p < 0.01), accompanied by no interaction in the Ponzo illusion group (F[1, 24] = 1.47). In the 3D-cube condition, planed comparisons showed a significant effect of the illusion for the VWM group (F[1, 13] = 17.48; MSE = 0.221; partial eta squared = 0.573; p < 0.01) with no effect for the iconic memory group. In the Ponzo illusion condition, planned comparisons showed significant effect of the illusion both for the VWM (F[1, 11] = 6.48; MSE = 0.627; partial eta squared = 0.37) and the iconic memory groups (F[1, 11] = 16.089; MSE = 0.627; partial eta squared = 0.593) with no interaction between the illusion direction and the memory store (F < 1). These findings provide converging evidence to the results of Experiments 1 and 2 by showing that, unlike between-objects contextual illusions that are represented in both VWM and iconic memory, only VWM produced within-object illusory effects with no illusory effects in the iconic memory condition. 

Figure 5

 

Effects of the rectangle's irrelevant depth (a) and the Ponzo illusion (b) on representations in iconic memory and in VWM in Experiment 3. These results replicate those found in Experiments 1 and 2, and provide converging evidence for the idea that within-object illusions have differential effects on iconic memory and VWM. Both iconic memory and VWM were affected by the between-objects Ponzo illusion.

Experiment 4

Experiment 4 was designed to verify that we were, in fact, tapping into iconic memory representations in the previous experiments. Previous studies have shown that informational persistence in iconic memory is not overwritten by a mask and can persist for about 600 ms following stimuli offset (Irwin & Yeomans, 1986; Smithson & Mollon, 2009). Therefore, it is unlikely that the masking used in our study has abolished the iconic representation. In our previous experiments, we masked our stimuli in order to tap informational persistence while disregarding visual persistence (Coltheart, 1980). Nonetheless, to provide stronger support for the idea that the mask used in previous experiments did not interfere with the iconic storage, we designed Experiment 4 to replicate the results of Experiments 1 and 2 without the presentation of a mask. In addition, although the time scales we used in the previous experiments were clearly within the time frame set by prior studies investigating representations in iconic memory (DiLollo, 1995; Irwin & Yeomans, 1986; Sligte & Lamme, 2008; Sperling, 1960; for reviews, see Coltheart, 1980; Irwin & Thomas, 2008; Long, 1980), it can still be argued that the relatively long (260 ms) duration between the disappearance of the target object and the presentation of the probe used in the iconic memory condition could have allowed participants to transfer the visual information to VWM. Such a strategy could have also been encouraged by the fact that target objects were presented in a fixed predicted location throughout the experiment. To exclude this possibility, in Experiment 4 we used shorter target-probe ISIs (150 ms) combined with location uncertainty. The location of the target was changed randomly from trial to trial and was either on the left or the right side of the fixation point. To the extent that the results of Experiment 4—in which a different criterion was used for iconic memory—would replicate those found in Experiments 1 and 2, it will provide support for our argument that the locus of the effect found in previous experiments can be attributed to iconic memory. 

Methods

Participants

Twenty-four undergraduate students received course credit or were paid for their participation in the experiment. A within-subject design was used, which included the iconic memory condition used in Experiments 1 and 2. All participants had normal or corrected-to-normal vision. Ethics were approved by the local ethics committee. 

Design and procedure

The design was similar to the iconic conditions used in Experiments 1 and 2 with three exceptions: First, a blank ISI of 150 ms was used between the disappearance of the target and the appearance of the probe. Second, no mask was presented following the presentation of the target stimuli. Finally, the target stimuli could have randomly appeared either 2.5 cm to the right or the left of the fixation. Block order (Ebbinghaus illusion block versus rectangle's width classification block) was counterbalanced between participants. 

Results and discussion

The goodness of fit of each subject to a logistic function was r² = .95 (std r² ± .05). Four participants were excluded from the data analysis due to their inability to perform the task, which resulted in low $r^2$ fits. The data was analyzed in a similar manner to the analysis used in Experiments 1 and 5 except for the addition of the within-subject factor of illusion type. The results are presented in Figure 6. As can be seen in the figure, the results replicated those found in Experiments 1 and 2. As in Experiment 1, the between-object illusion affected iconic representation with circles surrounded by larger circles perceived as smaller compared to circles surrounded by smaller circles. However, replicating the results of Experiment 2, the within-object illusion did not affect iconic representations. The data were submitted to an analysis of variance (ANOVA) with illusory context (perceptually large or perceptually small) and illusion type (within- or between-object relations) as within-subject variables. A significant interaction was found between the illusion context and illusion type (F[1, 16] = 5.545; MSE = 1.066; partial eta square = 0.257; p < 0.05). Further analysis revealed that the interaction resulted from a simple main effect of perceived size in the between-objects illusion condition (F[1, 16] = 5.008; MSE = 1.64; p < 0.05) but not in the within-object condition (F[1, 16] = 1.12). 

Figure 6

 

Effects of the Ebbinghaus illusion and the rectangle's irrelevant height on representations in iconic memory in Experiment 4. These results replicate those found in Experiments 1 and 2, and provide further support that the locus of these effects is the iconic memory storage.

The results further strengthen the notion that the locus of the effects found in the previous experiments can be attributed to iconic memory. The results of the present experiment provide converging evidence that representations in iconic memory are differently affected by within-object and between-objects illusions. 

General discussion

The results of Experiments 1 through 4 provide new evidence showing dissocations as well as associations between-object representations in iconic memory and in VWM. Iconic memory representations were affected by between-objects illusions but not by within-object illusions whereas VWM representations were affected by both types of illusions. Our findings that certain Gestalt aspects of perception are common to both VWM and iconic memory while others are not suggest that iconic memory and VWM contain distinct representations of object shape. The findings that iconic memory and VWM had similar effects on between-objects contextual illusions suggest that iconic memory cannot be categorically defined as part of a hierarchal buildup of information in a bottom-up fashion. Moreover, these findings show that certain aspects of holistic processing can be traced even to initial stages of perception. These results can be taken to support the reverse hierarchy model of perception (Hochstein & Ahissar, 2002), according to which early and automatic visual processing mechanisms are capable of fast parallel processing in which the global “gist” of the visual scene is initially extracted in a top-down fashion (for a similar idea, see Weisstein & Harris, 1974). Considering this model, iconic memory representations can be seen as a mixture of bottom-up processing and top-down processing achieved in a reverse hierarchy fashion that allows fast recognition of contextual relations between objects. The processing required for extracting the relations within objects could be considered as a more advanced processing stage. 

Note that, although our data support the idea of different representations in iconic memory and VWM, it can still be argued, given the vast and complicated literature related to their memory storages, that the new paradigm we used does not clearly distinguish between iconic memory representations and VWM representations. Although Experiment 4 was designed to directly address this issue, further research is needed to clearly disentangle the effects of iconic memory from the effects of subsequent processing stages. 

In the context of our discussion on the properties of iconic memory, a relevant distinction has been made between two modes of processing in the visual system; feed-forward sweep (FFS) and recurrent-processing (RP) (Lamme, 2010; Lamme & Roelfsema, 2000). FFS refers to the activation of hierarchal level of processing in the visual system connected in a feed-forward manner, starting from the lower and advancing to the higher visual cortical areas. RP, on the other hand, represents the cortical levels that remain active after their participation in the FFS via horizontal connections and feedback connections between layers. For example, while neurons in V1 can be tuned to a specific orientation and color, their responses can be also sensitive to contextual information that lands outside of their visual field (for a review, see Lamme & Roelfsema, 2000). In the iconic memory conditions in all experiments, the time between onsets of the target to the onsets of the probe were shorter than the suggested time course of full RP. As a consequence, iconic memory representations should have contained less RP information compared to VWM representations. Thus, our results could reflect the relationships between early and advanced RP, respectively. It is possible, therefore, that early RP extracts relational properties between separate objects, while more advanced stages of RP are needed in order to fully process the relational properties within an object. 

Several additional explanations can account for why iconic memory was not affected by within-object illusions in our study. One possibility is that iconic memory does not represent local features of objects and that further processing is required in order to extract those features. In other words, iconic memory is less sensitive to changes inside an object than VWM due to shallower processing. This explanation is in favor of the reverse hierarchy model as it suggests that the visual system initially extracts the global context of the scene and that the processing of featural information is achieved at a higher level of processing. It is also possible, however, that iconic memory represents featural information about objects alongside global information about the relationship between different objects. Such a hybrid representation cannot be fully accommodated by the reverse hierarchy model. Such explanation suggests that representations in iconic memory contain top-down as well as bottom-up information in which certain Gestalt aspects of the scene are processed by early RP alongside the availability of veridical information for FFS. The possibility that a perceiver can be accurate at the local (within-objects relations) level while being inaccurate at the global (between-objects relations) level of the same object fits more with the bottom-up account than a reverse hierarchy account. 

Our results show that visual illusions can be used as a powerful tool to probe the nature of representation within different stores of visual memory. We suggest that, while some features of holistic processing can be dissociated between iconic memory and VWM, information processing in iconic memory and VWM cannot be considered as a simple, bottom-up buildup of information as suggested by current models of visual processing (Coltheart, 1980; DiLollo, 1995; Irwin & Yeomans, 1986). 

Acknowledgments

We thank Joseph Tzelgov and Hilla Jacobson-Horowitz for their help and comments throughout this study. 

Commercial relationships: none. 

Corresponding author: Tzvi Ganel. 

Email: [e-mail]. 

Address: Department of Psychology, Ben-Gurion University of the Negev, Beer-Sheva, Israel.