Global statistics are not neglected

Abstract

In the framework of Reverse Hierarchy Theory it was suggested that initial vision at a glance brings the gist of the scene to conscious perception using explicit high cortical level representations, which are initially built by implicit bottom-up processing (Hochstein & Ahissar, 2002). Only later return to lower cortical level representations introduces local details to conscious perception. Global statistics of similar elements are perceived rapidly and accurately, suggesting they are included in the initial perception of the gist of the scene, not depending on prior conscious perception of local details. Patients with unilateral spatial neglect have difficulty responding to elements in their contralesional hemifield. However, this deficit is especially pronounced for tasks that require focused attention, i.e., are dependent on the reverse-hierarchy return. We review recent studies that indicate that perception of global statistics is among the spread attention tasks that are somewhat spared from this deficit. Combining these results, we suggest that perhaps the function of global statistics perception might include serving as a basic percept required for finding salient deviants from the mean, as in rapid odd element feature search paradigms, and perhaps subsequently focusing attention to them.

Introduction

A dozen years ago, Hochstein and Ahissar revolutionized our understanding of visual perception. In their seminal paper, they accepted the hierarchical view of initial visual information processing, but suggested that conscious perception follows a reverse hierarchy route (Hochstein & Ahissar, 2002). They demonstrated that, at first, we are consciously aware only of the high-level results of this first-pass processing hierarchy, without awareness of their detailed antecedents. Thus, the gist of the scene is perceived at first glance, by consciously attending to high-level global representations (Biederman, Rabinowitz, Glass, & Stacy, 1974). In contrast, only more lengthy vision with scrutiny adds local detailed information. This addition is accomplished by conscious perception returning to the appropriate lower-level representations, where detailed information is already present (Hock, Schöner, & Hochstein, 1996; Crick, & Koch, 1998). The reverse hierarchy return is guided in top-down fashion by preliminary results found at high cortical levels, and serves to corroborate (or correct) preliminary approximations, in addition to adding details to the perceived scene. Reverse Hierarchy Theory is schematically shown in Figure 1. 

Figure 1

 

Schematic Diagram of Classical Hierarchy and Reverse Hierarchy Theory (reproduced with permission from Hochstein & Ahissar, 2002). Classically, the visual system was seen as a hierarchy of cortical areas and cell types. Neurons of low-level areas (V1, V2) receive visual input and represent simple features such as lines or edges of specific orientation and location. Their outputs are integrated and processed by successive cortical levels (V3, V4, medial-temporal area MT), which gradually generalize over spatial parameters and specialize to represent global features. Finally, further levels (inferotemporal area IT, prefrontal area PF, etc.) integrate their outputs to represent abstract forms, objects, and categories. The function of feedback connections was unknown. Reverse Hierarchy Theory proposed that the above forward hierarchy acts implicitly, with explicit perception beginning at high-level cortex, representing the gist of the scene on the basis of a first-order approximate integration of low-level input. Later, explicit perception returns to lower areas via the feedback connections, to integrate into conscious vision with scrutiny the detailed information available there. Thus, initial perception is based on spread attention (large receptive fields), guessing at details, and making binding or conjunction errors. Later vision incorporates details, overcoming such blindnesses.

Hochstein and Ahissar (2002) based their conclusions on a number of striking phenomena (see also Pascual-Leone & Walsh, 2001; Di Russo, Martinez, & Hillyard, 2003; Juan, Campana, & Walsh, 2004; Ahissar, Nahum, Nelken, & Hochstein, 2009; Ishizu, Ayabe, & Kojima, 2009; Pavlovskaya & Hochstein, 2011). First of all, it had already been found that perceptual learning follows the same reverse hierarchy route (Ahissar & Hochstein, 1997, 2004). Other supporting phenomena included evidence of extremely rapid perception of the gist of the scene, for example when looking for a target in a series of images presented in Rapid Serial Visual Presentation (RSVP; Potter, 1975; Keysers, Xiao, Foeldíak, & Perrett, 2001; Kirchner, & Thorpe, 2006; Crouzet, Kirchner, & Thorpe, 2010). At the same time, numerous studies had shown that the details of the scene are not adequately perceived, leading to perceptual blindness. These findings included repetition blindness (Kanwisher, 1987, 1991; Luo, & Caramazza, 1996), change blindness (Rensink, O'Regan, & Clark, 1997), inattentional blindness (Mack, & Rock, 1998) and boundary extension (Intraub, 1999), where in each case the gist is perceived but the details are not—including the number of times a categorical element has been presented, presence of a change in a noncategorical fine detail of a scene, or the precise object-obstructing boundary of the scene. Another instance of inadequate perception is the attentional blink, where attempt to perceive an image detail prevents perception of a closely following image. 

An important corollary of the Reverse Hierarchy Theory relates to Anne Treisman's feature search versus conjunction search dichotomy. Treisman and Gelade (1980) suggested that the difference between these two types of search is that fast pop-out occurs when an element differs categorically in at least one dimension from other elements in an array, reflecting parallel search within a separate map for the critical dimension. As with other slow serial search, conjunction search, which occurs when there is a need to conjoin information from two maps, can only be accomplished by serial search, position by position, within the visual field. Disregarding the details of the model, the essential difference between these two search strategies lies in the use of focused attention for slow search and global spread attention for pop-out (called preattention, in Treisman's original terminology). The reverse hierarchy conclusion is that fast feature search based on global attention (i.e., use of large receptive fields) must be a high cortical level phenomenon, whereas serial search, based on local focused attention (and therefore small receptive fields), must rely on a slower reverse hierarchy return to lower cortical areas. Note that global spread attention and large receptive fields are not synonymous with sensitivity to low spatial frequencies, as the large fields might respond to high spatial frequencies throughout, or anywhere within, the large receptive field. Reverse hierarchy theory suggested that rapid perception is high level and slower perception is low level. In support of this counterintuitive conclusion, Hershler and Hochstein (2005, 2006, 2009; see also Hershler, Golan, Bentin, & Hochstein, 2010) found that faces pop-out from heterogeneous arrays of images. Since face perception must be high level (presumably involving the FFA fusiform face area), face pop-out supports the conclusion that pop-out is generally high level. 

Other perceptual phenomena also support the high level first theory. For example, there has been a decades-long debate over the cortical site of binocular rivalry. If the rivalry is between the representations deriving from the two eyes, it would seem that rivalry must be at V1 or earlier, since information from the two eyes is unified in V1, and higher cortical levels do not differentiate utrocular information. On the other hand, rivalry could be between competing interpretations of the visual scene, independent of its deriving from two eyes or other splits. Support for high-level rivalry comes from a number of sources. Ilona Kovács and her colleagues found that when two images are intermixed in a patchwork form, with complementary patches presented to the two eyes, perception alternates between the full coherent images, not between the patchworks presented to each eye (Kovács, Papathomas, Yang, & Fehér, 1996). Similarly, Logothetis and colleagues found that rapid swapping of images between the two eyes (at 1–3 Hz), that is, a rapid transfer of the source of information between the eyes, results in slow perceptual alternations between the images, i.e., rivalry between interpretations rather than between eyes at normal rates (0.3–0.5 Hz; Logothetis, Leopold, & Sheinberg, 1996). Wolf and Hochstein (2011) found that when one eye sees words and the other nonwords, there is a difference in rivalry dominance times. The difference can only derive from a high level mechanism, which knows how to distinguish words from nonwords. Rubin found similarities between binocular rivalry and other bistable phenomena, suggesting they have the same high-level source (Rubin, 2003; Rubin, & Hupe, 2005). Early support for high-level rivalry came from an amazingly prescient paper of Anne Treisman (1962) who found that when each eye is shown a differently colored circle (e.g., red in the right eye and green in the left), which is displaced differently in each eye relative to a surrounding (black) circle, observers experience rivalry between the two colors, but retain the binocular depth disparity percept. Thus, disparity information is taken into account even while the stimulus color is suppressed, and rivalry must be, or at least involve, a high-level mechanism. 

Statistical properties

We come to the focus of the current review, perception of statistical properties of a presented image (size: Ariely & Burbeck, 1995; Ariely, 2001; Chong & Treisman, 2003, 2005a, 2005b; Corbett & Oriet, 2011; orientation: Dakin & Watt, 1997; Parkes, Lund, Angelucci, Solomon & Morgan, 2001; Alvarez & Oliva, 2009; brightness: Bauer, 2009; location: Alvarez & Oliva, 2008; motion speed and direction: Watamaniuk & Duchon, 1992; Watamaniuk, Sekuler, & Williams, 1989; face gender, emotional expression, and identity: Haberman & Whitney, 2007, 2011; de Fockert & Wolfenstein, 2009). 

As suggested by many papers in the present special issue, statistical property perception is not secondary, dependent on prior perception of individual element properties, but a first-order percept integrating global information (e.g., Corbett & Oriet, 2011). It has been suggested that when focal attention is unavailable, summary statistics compensates somewhat for the lack of detailed perception (Alvarez & Oliva 2008, 2009), so that observers in a difficult tracking task were better at maintaining mean location of an unattended group than the location of any single item, and background orientation changes were noted when their summary statistics changed, as well. The availability of summary statistics may also be the source of the compelling impression that we have a complete and accurate picture of our world, despite not perceiving details of individual objects present (Haberman & Whitney, 2007, 2012). It has been suggested that alongside of individual object vision, which has a very limited capacity (Miller, 1956; Luck, & Vogel, 1997; Cowan, 2001; Halford, Baker, McCredden, & Bain, 2005), statistical property perception serves as a separate mechanism affording analysis of groups of similar elements, thereby overcoming this limitation (Lanzoni, Melcher, Miceli, & Corbett, 2014). 

These suggestions hint that perception of summary statistics is a high-level phenomenon, which integrates low-level information that is itself lost to conscious perception. We know the mean without knowing (consciously) the individual values. Thus, it was found that included in the calculation of average size are even items that are not available to conscious perception (due to masking: Choo & Franconeri, 2010; crowding: Parkes et al., 2001; or not being attended focally: Joo, Shin, Chong, Blake, 2009; Corbett & Oriet, 2011). Dramatic evidence in this direction comes from experiments testing change blindness at the same time as summary statistics. It was shown that even though observers were unable to localize the position of items which changed (faces whose expression changed), nevertheless these items were included in the processing of summary (expression) statistics (Haberman & Whitney, 2011). All these also counter the proposal that statistical properties actually depend on only a few items in the display (Myczek & Simons, 2008; Marchant & de Fockert, 2010; see Chong, Joo, Emmanouil, & Treisman, 2008). 

In terms of reverse hierarchy theory, statistical property perception is the result of early implicit feed-forward processing assembled in high-level representations with global attention. This conclusion suggests that statistical property perception should not require focused attention—in fact, focused attention might even interfere with accumulation of global property percepts. 

The corollary to this conclusion that we review in this paper is that individuals who are challenged in focusing attention to part of the visual field might nonetheless be able to include items in these “neglected” regions in their statistical property analysis. We review studies of patients with unilateral neglect syndrome, providing evidence for the inclusion of neglect field elements in the perception of statistical properties. This review also adds to the accumulating evidence that unilateral neglect only affects focused attention perception, and not global “pre-attention” perception. 

Unilateral spatial neglect

Unilateral spatial neglect (USN) is a common neurological disorder caused by unilateral brain damage, usually in the right hemisphere, characterized by a failure to perceive and explore stimuli on the contralesional side, i.e., usually on the left (Vallar, & Perani, 1986, 1987; Heilman, Watson, & Valenstein 1993; Mesulam, 1999; Robertson, & Rafal, 2000; Kerkhoff, 2001; Mort et al., 2003; Corbetta, Kincade, Lewis, Snyder, & Sapir, 2005; Bonneh et al., 2008; Kaufman, Serfaty, Deouell, Ruppin, & Soroker, 2009; Corbetta, & Shulman, 2011; Oren, Soroker, & Deouell, 2013; concerning related representational neglect, see Salvato, Sedda, & Bottini, 2014). USN is attributed to the inability of focusing attention, preventing left-side inputs from reaching conscious awareness and eliciting orienting behavior (Posner, Walker, Friedrich, & Rafal, 1987; Kinsbourne, 1993; Deouell, Sacher, & Soroker, 2005; for a review, see Driver & Vuilleumier, 2001). 

A common finding among patients with USN is the phenomenon of extinction, in which patients disregard a contralesional stimulus in conditions of simultaneous bilateral stimulation, though the same stimulus may be detected and reported when presented in isolation. Recent studies found a double dissociation between neglect and extinction, suggesting that these two conditions may reflect separate dysfunctions (Pavlovskaya, Sagi, Soroker, & Ring, 1997; Pavlovskaya, Soroker, & Bonneh, 2007; (Pavlovskaya, Soroker, & Bonneh, 2007; Umarova, et al., 2011; Vossel et al., 2011; for a review, see de Haan, Karnath, & Driver, 2012). 

Not all tasks require focused attention. In fact, many tasks may be accomplished so rapidly that there is insufficient time for serially focusing attention to the variety of local scene attributes. As discussed above, feature search pop-out is one such example, and reverse hierarchy theory suggested an anatomical-physiological separation between spread versus focused attention mechanisms. Regarding USN, if neglect impairs mainly focused attention mechanisms and related perception of local features (Pavlovskaya, Ring, Groswasser, Keren, & Hochstein, 2001), we might expect USN to spare somewhat perception of global properties. Indeed, it was found that USN patients have a much greater deficit for conjunction search, which depends on serial focused attention, than for feature search, which requires only spread attention, especially when studied using arrays that include elements on both sides of the midline (presumably including extinction), as demonstrated in Figure 2 (data from Pavlovskaya et al., 2002; see also Riddoch, & Humphreys, 1987; Robertson, Lamb, & Knight, 1988; Eglin, Robertson, & Knight, 1989, 1991; Robertson, Eglin, & Knight, 2003; Esterman, McGlinchey-Berroth, Milberg, 2000; Laeng, Brennen, & Espeseth, 2002; Hildebrandt, Schütze, Ebke, Brunner-Beeg, & Eling, 2005; List et al., 2008; Erez, Katz, Ring, & Soroker, 2009; Van Vleet & Robertson, 2009). Thus, neglect results in poorer left side feature search, implying that even rapid search requires attention, but only global spread attention. There is a strikingly greater neglect effect for search that involves serial focused attention, in which USN patients have trouble focusing attention to each element serially. 

Figure 2

 

Performance of unilateral spatial neglect patients on rapid global feature search (blue) compared to slower, serial conjunction search (adapted, with permission, on the basis of data of two participants from Pavlovskaya, Ring, Groswasser, & Hochstein [2002]. Searching with unilateral neglect. Journal of Cognitive Neuroscience, 14, 745–756.). Performance shows a distinct left-side deficit for both search types, but is much more severe for serial focused attention conjunction search. Position on array is in terms of number of array elements where the array was presented centrally and elements were 0.75° apart.

Interestingly, it was found that flanking elements, added on the contralesional hemifield side of a feature-search array actually aided search, presumably by grouping with array elements and redefining its reference frame (Grabowecky, Robertson, & Treisman, 1993). Laeng and colleagues found in a study of a single patient that neglect responses (misses on left-side target-present trials) were speeded for feature search (but not for serial search) perhaps reflecting pop-out detection without subsequent attentional confirmation (Laeng et al., 2002). Recently, Moreh and colleagues tested immediate recall of the names of four simultaneously presented images as well as their later recognition memory. They found a strong left side disadvantage for USN patients in both immediate naming and later recognition. Significantly, correctly named objects were also recognized later, though sometimes even nonnamed, neglected objects produced recognizable memory traces (Moreh, Malkinson, Zohary, & Soroker, 2014; see also Haramati, Soroker, Dudai, & Levy, 2008). Presumably, categorical image naming and recognition are largely global perception effects, which are somewhat retained in cases of USN. Similarly, in an object recognition task, USN patients neglected left side elements of objects based on their visual mass centroids rather than their spatial midline (Pavlovskaya, Glass, Soroker, Blum, & Groswasser, 1997). Computation of such centroids at an early stage of visual processing seemingly is based on extracted statistical properties. 

USN and statistical properties

A first hint that statistical properties may be perceived, despite cortical damage leading to compromised focal attention, came from a study of a patient with simultanagnosia (G. K.; Demeyere, Rzeskiewicz, Humphreys, & Humphreys, 2008). Even though the patient had difficulty reporting perception of more than a single item, he was aware of the mean size and mean color of the unperceived items, and confused between the presence of a sample and the mean of all the presented items. 

Yamanashi Leib and coauthors found that when USN patients performed a search task, there was some implicit averaging of size information in the left hemifield, which actually interfered with performance (Yamanashi Leib, Landau, Baek, Chong, & Robertson, 2012). They showed USN patients a single target circle in the center of the screen for 500 ms with a size that varied from trial to trial. This was followed immediately by a search display containing a group of circles on one side and a group of task-irrelevant triangles on the other, as shown in Figure 3, top (adapted from Yamanashi Lieb et al., 2012). Patients were instructed to ignore the triangles and to indicate whether the target size was present or absent in the group of circles, by verbalizing “yes” for present or “no” for absent. Half of the targets matched the mean size of the group of circles within the search display, and half did not. As expected for USN patients, hit rates were higher when the arrays with a target were presented on the right side compared to the left. The interesting finding was that when the target was absent, the responses of most patients were influenced by the mean size of the circles. They were more likely to (incorrectly) say it was present, if the mean was equal to the size of the searched-for target than when it was not equal. This finding was especially striking, as the opposite was true when the circles were on the right; that is, there were more false alarms when the searched-for target size was not equal to the nonmean than when it was. These results are shown in Figure 3, bottom. Furthermore, these patients found it harder to disregard left side elements when searching on the right. In summary, the conclusion of this study was that USN patients implicitly compute the mean size of circles or triangles on either side of the display, and this computation affects their search for a single circle with the same size as the probe presented to them at the beginning of the trial. 

Figure 3

 

(Adapted with permission from data of Yamanashi Leib, Landau, Baek, Chong, & Robertson [2012]. Extracting the mean size across the visual field in patients with mild, chronic unilateral neglect. Frontiers in Human Neuroscience, 6, 267.).Top: Examples of images shown to participants. First a reference circle, and then a group of circles of different sizes, where a circle matching the reference was sometimes included. Participants were asked if a circle of the size of the reference was present in the array. Bottom: USN patients performed significantly better, that is, had many more target-present hits for right side circle arrays than for left side arrays (left bars). However, when the mean circle size was equal to the reference circle size, patients had many more target-absent false alarms for left side arrays (middle bars), but the opposite occurred when the mean size was not equal to the reference (right bars). Thus, even USN patients must have been computing implicitly the mean circle size.

Recently, another study confirmed that neglect patients implicitly perform statistical processing even in their neglect field. Lanzoni and colleagues found that patients searched more efficiently for an odd orientation Gabor when the mean distractor size remained constant across trials (Lanzoni et al., 2014). 

We studied 12 USN patients to directly determine if they compute mean size in their neglected field (Pavlovskaya, Soroker, Bonneh, & Hochstein, 2015; see this text for full description of the conditions and ANOVA statistical significance of the results). We explored the explicit inclusion of left side elements in computing average size of an array that spanned both sides of the display. We wished to elucidate further the question of processing asymmetry in USN, using the task of explicit average size computation as a probe. Specifically, we asked whether right-hemisphere damaged subjects with unilateral spatial neglect, in whom allocation of focused attention to left-sided stimuli is markedly impaired, include objects on the left in a size-averaging task. This was the result expected on the basis of the preattentive account of statistical processing (Chong & Treisman, 2003, 2005a, 2005b). In contrast, if the averaging task were performed by serial, attention-demanding weighting mechanisms, which are deficient in USN patients, we would have expected total disregard of the contribution of left side objects to the group average. We used a variety of experimental conditions related to the complex nature of the neglect phenomenon, including the division between neglect and extinction (Pavlovskaya, Bonneh, Soroker, & Hochstein, 2010, 2011; Pavlovskaya et al., 2015). 

The most enlightening case was when we presented a large array of circles, which extended well into the left and right sides of the display. Participants were asked to judge if the mean size of the circles was larger or smaller than a previously viewed reference circle. However, we carefully manipulated the array circle sizes so that the average size on the left and the average size on the right were not necessarily the same. One example is shown in Figure 4, where the mean size on the left is larger than that on the right. 

Figure 4

 

An example of the array of circles shown to participants in the experiments of Pavlovskaya, Soroker, Bonneh, & Hochstein, 2015; drawn with permission. Note that the mean size is different on the left (larger) and on the right (smaller). Participants used both sides in their computation, with even USN patients including left side circles—through with lesser weight.

If the means on the two sides were the same, participants could use either side or both to perform the task. But when they were different, we set the average on one side to have a difference four times greater from the reference than did the other side. Thus, the best strategy would have been to compute only the average on that side. However, participants were not told which side was the more informative, so they had no choice but to use the mean of the entire array. On the other hand, USN patients might be expected to disregard the circles on the left, and perform better than controls when the more informative circles were on the right, and much worse when the informative ones were on the left. 

This expectation is not what we found. Instead, even the group of 12 patients included left side circles in their average computation. However, they gave these circles less weight than those on the right. The top bar graph of Figure 5 compares performance for controls (left) and patients (right). For controls, there is no significant difference between performance with the left or the right side being more informative, and when both are equally informative, performance is somewhat better. Patients demonstrate a very different result. For them performance is considerably worse when the left side circles are more informative, and there is less difference between right side and both sides being informative. But the right side to both sides difference is still present and significant, suggesting that left side circles do enter into their computation. 

Figure 5

 

(reproduced with permission from figure 8 of Pavlovskaya, Soroker, Bonneh, & Hochstein [2015]. Computing an average when part of the population is not perceived. Journal of Cognitive Neuroscience, 27(7), 1–15.). Results for healthy controls (left) and USN patients (right) in different conditions with bilateral arrays, that is, with the more informative circles on the left, on the right, or on both sides (circles on the other side had an average of one quarter the difference in size from the reference circle). Results are shown with concordant (top) differences (both sides in the same direction from the reference, either both larger or both smaller than the reference) and with discordant (bottom) differences (one side's average size larger and the other side's average smaller than the reference circle). Introducing discordant average sizes exaggerates the trends seen with concordant sizes. Performance in the left and right conditions are considerably poorer compared to the both condition even in controls. For patients, performance is much worse for left than for right conditions. However, performance for the right condition is not the same as the both sides condition, showing that patients, too, take into account the circles on the left. Patients have especially poor performance in the discordant left condition, which is actually below 50%, meaning that, when the averages on the two sides are in opposite directions compared to the reference circle, they depend more on the smaller right side average delta than on the larger left side average delta. Nevertheless, the smaller (and incorrectly signed) delta on the left side still affects performance and it is poorer in the right condition than in the both condition.

In a second condition, called discordant, we had the average circle size on the two sides of the display differing from the reference circle in opposite directions. This condition was meant to make the task still more difficult. Again, however, the average on one side differed from the reference by four times as much as the other side. Thus, if the average on one side was much larger than the reference, the average on the other side was a bit smaller than the reference. Here, too, controls could only average the circles on the two sides, and the misinformation on the second side is expected to diminish performance even more than in the concordant condition (where both sides differ from the reference in the same direction). This result is seen in Figure 5, bottom left, where right and left side performance is considerably worse than for the concordant condition (Figure 5, top left). 

Patients might be expected to do much better than controls when the informative side is on the right (since they might disregard the misinformation on the left) and to actually get it always wrong when the informative side is on the left—if they totally neglect this side. Results show, however, that patients again took both sides into account, and again gave less weight to the left side. Whereas patients perform less well even when both sides are equally informative, they are actually better than controls when the informative side is on the right—since they give less weight to the misinformation on the left. When the informative side is their neglected left side, they should always give the wrong answer, if they were to only take the right-side circles into account. In fact, they are correct a bit less than 50% of the time, but not 20% or 30%, meaning they are including the left side circles in their statistical property computation. 

We took one more step in our study. We varied the difference between the average and the reference circle, changing the difficulty of the task, and measured performance as a function of this difference. Figure 6 shows the result for controls (left) and patients (right), and for the concordant (top) and discordant (bottom) conditions. In each graph, three plots show the results for more informative left (blue) or right (red) side, and for both sides equally informative (green). Note the large improvement for controls when both sides are informative, which is still larger for the discordant condition. For the concordant condition, right side or both sides are closer for patients, since the left side contributes relatively little to the computation. Still, the plot for right side is somewhat lower than that for both sides contributing equally. Dramatically, the patient data plot for left side informative circles actually gets worse as the difference grows and the right side misinformation dominates. Nevertheless, the left side contribution remains present, here, too. 

Figure 6

 

(Reproduced with permission from figure 10 of Pavlovskaya, Soroker, Bonneh, & Hochstein [2015]. Computing an average when part of the population is not perceived. Journal of Cognitive Neuroscience, 27(7), 1–15.). Performance as a function of delta (the difference between the reference circle size and the average circle size on the informative side). Average size computation in bilateral arrays with concordant (top) or discordant (bottom) delta directions. Performance is better when the circles on both sides are informative (green) relative to trials where one side is less informative (right, red; left, blue). For controls, the both-minus-right and both-minus-left differences are equal; but for USN patients the latter difference is much bigger, reflecting a relatively smaller weight given to left circles. Evidently, left-side circles do contribute to the average computation even with USN. The trend seen for concordant deltas is exaggerated for discordant deltas with a more notable disadvantage for left compared to right in the USN group. Note that for patients and the left discordant condition, increasing delta augments right side circle contributions, leading to increasingly poorer performance. This is the only case of performance below 50%, with participants judging most arrays as in the direction of the smaller side delta and performance decreasing rather than increasing with increasing delta.

We conclude that for spread attention tasks, including average size computation, left side information is gathered and makes a contribution to visual perception. This contribution is less than that of the right side, but it is present in any case. 

Discussion

We have framed this review in terms of Reverse Hierarchy Theory, showing that many of the characteristics of statistical property perception are consistent with this theory. Other theoretical frameworks may also be invoked to test their compatibility with these characteristics. The important point is that summary statistics are perceived as part of the gist of the scene or its Gestalt. It has long been known that the Gestalt is consciously perceived before the details of the environment or its objects, and the question has been raised how this is done. Focusing on summary statistics may help solve this puzzle, and Reverse Hierarchy Theory is only one possible framework. 

For example, Wolfe and colleagues have suggested a series of guided search (GS) models, whereby a preliminary parallel “pre-attentive” search may not result in pop-out, but may nevertheless guide a further serial search with attention (GS1: Wolfe, Cave, & Franzel, 1989; GS2: Wolfe, 1994; GS3: Wolfe, & Gancarz, 1996; GS4: Wolfe, 2007). Whereas Wolfe has not to our knowledge applied GS theory specifically to preliminary summary statistics, one could imagine that the first parallel search could include this information and use it as a guide for the next stage focal attention search. Interestingly, the computed mean also primes further perception of a single item matching this mean, adding to its salience (Marchant & de Fockert, 2009). 

Similarly, a recently proposed contextual guidance model suggests that even without explicit awareness, global scene context information may benefit object search mechanisms by modulating the use of the features provided by local image analysis (Torralba & Oliva, 2003; Torralba, Oliva, Castelhano, & Henderson, 2006). In the contextual guidance model, contextual information is integrated prior to even a first saccade, reducing the number of image locations that need to be considered by object-driven attentional mechanisms. The local pathway represents each spatial location independently. Two parallel pathways are seen to analyze the scene: A local representation computes image saliency, performing object recognition on the basis of local appearance, and a global pathway represents the entire image holistically by extracting global statistics from the image. The global pathway provides information about the expected location of the target in the image, as in other models with parallel, rather than sequential pathways (Neumann, Schweinberger, & Burton, 2013; De Fockert, & Marchant, 2008). As Torralba and colleagues point out, early impact of scene context is also compatible with reverse hierarchy theory (Hochstein & Ahissar, 2002) in which properties that are abstracted late in visual processing (such as object shapes, categorical scene description) rapidly feed back into early stages and constrain local processing. 

Why do we compute statistical properties? A proposal

There are important implications deriving from this analysis of neglect and statistical property perception, which allow us to return to understanding the basis of different search phenomena. It has been suggested that global statistical property perception serves to overcome the lack of focal attention to specific items, giving the impression of a complete picture of the world, and overcoming the limited capacity of object perception (see citations in the introductory Statistical Properties section above, including Haberman & Whitney, 2007, 2012; Alvarez & Oliva, 2008, 2009). It has been suggested that in contrast to individual object vision, which has a very limited capacity (Miller, 1956; Luck, & Vogel, 1997; Cowan, 2001; Halford et al., 2005), statistical property perception serves as an additional mechanism affording analysis of groups of similar elements, thereby overcoming this limitation (Lanzoni et al., 2014). Furthermore, the computed average of many imprecise measurements may be much more accurate than each individual measurement, when their sources of noise are not correlated, so that ensemble representation may serve to add accuracy (Zohary, Hillman, & Hochstein, 1990; Zohary, Shadlen, & Newsome, 1994; Alvarez, 2011). 

Another good reason for computing averages might be to facilitate the detection of deviants (Cavanagh, 2001). The dichotomy suggested by Anne Treisman and colleagues between rapid parallel and slow serial search is not limited to feature versus conjunction search. In general, search will be slowed if the difference between the target and the distractors is small, as demonstrated in Figure 7. (The line that differs by 45° from the other lines pops out on the left, but not on the right where jitter is introduced among the distractors.) But how small is small? We suggest that small or large difference depends on the perceived dispersion of the distractors, themselves. Even when presenting uniform distractors, as most studies have done, visual system noise will always result in perceived variations. 

Figure 7

 

An array with two targets, with no distractor jitter on the left, and considerable jitter on the right. Only the left side target pops out.

In natural visual scenes, extraction of statistical information (i.e., an estimate of average contrast, luminance, size etc.), using spread attention, is likely to operate on a regional basis, in different segments of the field of vision. This initial processing yields a rough parcellation of the visual scene where elements whose attributes are close to the regional mean value are represented as unsegregated parts of the region (e.g., leaves within a tree). On the other hand, elements whose attributes deviate significantly from the regional mean become targets of subsequent search processes verifying their meaning. Serial orienting of focal attention to the corresponding spatial locations enables feature identification and elaborate description of the scene and its constituents. 

We propose that the following steps might be taken by the visual system (not necessarily sequentially): (a) rapid, whole-field screening of the scene using spread attention; (b) statistical property processing including computation of the average size, contrast, etc., as well as their standard deviation or another measure of the variation or dispersion about the mean—again using global spread attention mechanisms; (c) mapping the location of salient deviants in the visual field, determined by their being different from the average element by more than a measure dependent on the dispersion about the mean. Deviation may be in any of the features measured, including size, contrast, color, shape, spatial frequency, object category, faced-ness, facial expression, and so on. Deviation leads to salience, perhaps represented on a salience map or a set of such maps; and (d) using the emerging salience map for mobilization of focused attention to the location of deviants. Thus, a deviant's location is the prime target for orienting of selective focused attention. Whereas the source of this deviation is bottom-up, the focusing itself depends on use of the salience map, and is thus a top-down process. 

The first two steps, which depend on global, perhaps whole-scene, processing, are more or less spared by the unilateral spatial neglect syndrome, and in particular its extinction phenomenon. USN patients might automatically include left side elements in their global screening of the scene and computation of scene statistical properties. They fail only in the subsequent focusing of attention to elements that deviate from the mean, especially when their attention is tied up with right side elements. We do not know, at this point, whether their visual systems detect the deviants (step 3), and the deficit is only in subsequent attending to them, or if they fail in (implicitly) detecting the deviants altogether. Further study is needed to address this issue and to measure the dependence of search efficiency on distractor variance in USN patients and controls. 

This proposal is consistent with that of Halligan and Marshall (1994; see also Robertson & Lamb, 1991) who suggested that visual neglect is not a selective impairment in attending to global form per se, but impairment in binding information from different levels of the perceptual hierarchy. Patients might access a global representation of a stimulus while at the same time being unable to use this information for subsequent performance of local tasks. The failure is in failed cross talk between spread attention mechanisms of the right hemisphere and focused attention mechanisms of the left hemisphere (i.e., a failure in step 4, above). Statistical data processing can be taken as a supporting evidence for this theoretical conjecture, placing the key deficit in USN at the stage where the output of early visual processing is necessary for bottom-up allocation of selective attention. 

Detection of deviants based on initial statistical property analysis is shared by other sensory modalities. Using mismatch negativity (MMN) as an electrophysiological marker of preattentive deviant detection in the auditory modality, it was found, in a group of USN patients, that the MMN was formed following exposure to left-sided pitch and duration deviants (but not from spatial-location deviants on the left; Deouell, Bentin, & Soroker, 2000). In the auditory modality the deviant attribute of a stimulus is judged against a background of average representation stored in a short-term buffer, formed by a sequence of discrete sound stimuli, and the MMN is elicited when the new stimulus deviates from the averaged representation. Thus, USN patients make use of left-sided auditory information for the purpose of preattentive deviant detection based on comparison to a stored computed average. 

Nevertheless, even with neglect, patients are quite good at feature search (Pavlovskaya et al., 2002) even on the left side, as demonstrated in Figure 2. Similarly it was found that when global averages remain constant, salient target processing is facilitated in neglected regions (Lanzoni et al., 2014). These results might stem from high-level target analysis without the need for top-down guidance. 

Thus, elaborating on the nature of the deficit underlying neglect with the aid of data concerning stimulus detection and statistical property computation, and analysis of how statistical properties of elements in the “neglected” field are included rather than excluded in this computation sheds light on salience mapping and the pop-out phenomenon, in general. It has probably been difficult to grasp the contribution of statistical processing to visual attention because researchers have been accustomed to use more or less uniform backgrounds in visual search experiments, whereas in real life one needs to segregate the target stimulus from the background of a much more heterogeneous array of stimuli, making statistical processing more important and relevant. 

Consider higher order statistical properties such as faced-ness and expression. There may be an important evolutionary advantage to have the ability of choosing a bush with many berries relative to other bushes with fewer fruit. And it may be important to be able to choose the one piece of fruit that is significantly riper than the surrounding pieces. For the social beings that we are, it must be advantageous to be able to choose a group of people with a welcoming expression relative to another that has, on average, a menacing look. It may also be important to detect the one smile among a uniformly angry crowd. Thus, statistical property perception might be a basic perceptual ability, especially when tied to a feature detection process. 

Returning to the neglect syndrome, it becomes clear why these patients suffer real life disadvantages being unable to use their statistical processing to direct their attention to salient deviants from surrounding boring elements. Leftward extension of the receptive fields of remaining right parietal neurons is likely to account for relative preservation of left-sided feature search. It is also likely to explain why size data of left-sided objects is taken into account in the entire field averaging procedure. 

Acknowledgments

We thank Anne Treisman, Lynn Robertson, and Merav Ahissar for discussions of issues on which this review is based. We are grateful to an anonymous reviewer and JOV editorial board member Jennifer E. Corbett for their comprehensive and helpful suggestions to an earlier version of this review. Supported by the Israel Science Foundation ISF. 

Commercial relationships: none 

Corresponding author: Shaul Hochstein. 

Email: [e-mail]. 

Address: ELSC, ICNC, Neurobiology Department, Life Sciences Institute, Hebrew University, Jerusalem, Israel.