Differential processing of invisible congruent and incongruent scenes: A case for unconscious integration

Abstract

Abstract

Abstract:

Abstract  Integration is held to be a key feature of conscious awareness. Some even argue that the latter cannot occur without the former. We tested this claim by presenting masked scenes depicting a person performing an action with a congruent or an incongruent object (e.g., a man pouring coffee into a mug or into a roll of toilet paper). The masked scenes were then followed by briefly flashed targets that again included a congruent or an incongruent object, and subjects were asked to judge targets' congruency as fast as possible. Reaction times (RTs) for targets preceded by perceptually invisible scenes that included an incongruent object were longer than RTs for targets that were preceded by congruent images. This implicit measure suggests that subjects processed certain relations between the object and its background—or at least between the object and another object in the scene—despite being unaware of either. Subjective and objective measures confirmed the invisibility of the masked scenes, ruling out partial awareness. These results suggest that incongruency between scene elements can be unconsciously processed even at impoverished presentation conditions, with reduced contrast and exposure durations as short as 33 ms. They provide evidence for ongoing contextual influences of unseen stimuli on the processing of a subsequent target.

Introduction

At every waking moment, we experience the external world in a unified manner. In what seems to be an effortless, natural process, we continuously integrate numerous types of information from various modalities into a coherent percept: our home, a street, the beach. Such complex integration of information—here, different objects and background into a single, visual scene—is considered by many to be the hallmark of conscious awareness (see, e.g., Baars, 2005; Dehaene & Naccache, 2001; Lamme, 2006; Lamme & Roelfsema, 2000; and most prominently, Tononi, 2013; Tononi & Edelman, 1998; Tononi & Koch, 2008). Most theories of consciousness hold unconscious processes to run in parallel, operating in encapsulated modules with few interactions with other processing units, while conscious processes are claimed to inherently involve long-range interactions between distinct brain regions (see, e.g., Crick & Koch, 2003; Dehaene & Naccache, 2001) through recurrent loops and feedback projections (Lamme & Roelfsema, 2000). These enable the rapid integration of signals from a variety of modalities and submodalities that is required for establishing specific relationships first between features (Treisman & Gelade, 1980) and then between individual items, until a structured mental representation is formed (Engel, Fries, Konig, Brecht, & Singer, 1999; Goodale, 2004). Thus, irreducibility—that is, the integration of information contained above and beyond that possessed by its individual parts—is considered the key property of consciousness (Tononi, 2004). 

Koch and Tononi (2011) have suggested how this property could be tested within the restricted domain of stationary visual scenes using operational tests for visual-scene consciousness. Building upon the assumption that consciousness is necessary for integrative processes, they reasoned that any system that showed integrative capacities should be considered as conscious. In fact, according to the integrated-information theory (Tononi, 2004, 2013), the higher the level of consciousness a system has, the higher is its ability to produce integrated information. Accordingly, and reminiscent of the traditional Turing test for machine intelligence (Turing, 1950), Koch and Tononi's test essentially amounts to asking arbitrary questions about what is going on in any one photograph. One way of doing so is by asking the system to distinguish “congruent” from “incongruent” images, such as an ice-skater on a rug in the living room, a transparent cow, or a cat chasing a dog. A system that could pass a great variety of such tests could be considered to possess a form of visual consciousness. 

We set out here to experimentally examine this operational test using conscious human observers. Translated into a psychophysical experimental paradigm, we tested the supposition that processing the incongruency of a scene can only be done when the image is visible and not when it is masked. Accordingly, we define integration here as the combination of two or more distinct features, or constituents, of the scene. Such a combination, according to the assumption that consciousness is needed for integration, can only be consciously performed. 

This assumption could be questioned; previous studies have demonstrated different forms of high-level processing in the absence of awareness (for review, see van Gaal, De Lange, & Cohen, 2012), some of which involve complicated integrative processes (e.g., combining words into congruent and incongruent sentences, see Sklar et al., 2012). However, these have usually used simple visual stimuli such as arrows (De Lange, Van Gaal, Lamme, & Dehaene, 2011), numbers (García-Orza, Damas-López, Matas, & Rodríguez, 2009; Ric & Muller, 2012; Sklar et al., 2012; Van Opstal, de Lange, & Dehaene, 2011), or words (Reber & Henke, 2012; Sklar et al., 2012). Only one study presented subjects with suppressed real-life complex scenes and directly demonstrated differential processing between congruent and incongruent images (Mudrik, Breska, Lamy, & Deouell, 2011), along the lines of Koch and Tononi's test. 

This study used the technique of continuous flash suppression (CFS; Tsuchiya & Koch, 2005; Tsuchiya, Koch, Gilroy, & Blake, 2006), in which distinct colorful patterns (“Mondrians”) are flashed successively at approximately 10 Hz into one eye, leading to a prolonged suppression of a lower contrast and stationary image presented to the other eye. Yet despite the long and powerful suppression, at some point the suppressed image does gain dominance over the Mondrian suppressor, thereby becoming visible to the observer. This point in time can serve as a measure of the level of unconscious processing during suppression (Jiang, Costello, & He, 2007): When images that systematically differ on some dimension break suppression at different times, one must infer that this difference was somehow processed and that this differential processing is associated with differential response times. 

Therefore, Mudrik, Breska, and colleagues (2011) measured suppression durations of congruent and incongruent visual scenes (e.g., a woman talking over a telephone vs. a woman “talking over” a shoe). Incongruent scenes emerged into awareness about 140 ms earlier than congruent scenes. This time difference was not found in a control condition, where the two types of scenes were presented binocularly on top of the Mondrians with gradually increasing contrast. As opposed to the experimental condition, the gradual binocular presentation does not involve a prolonged period of complete invisibility but only a stage of partial awareness (Kouider & Dupoux, 2004) leading to full awareness. Hence it was concluded that integration of the scene's elements took place during unconscious processing rather than partial awareness. This finding thus challenges the notion that integrating the image of an individual carrying out an action with an object appropriate to the image's background depends on conscious perception of this image (Koch, 2011), as in the Koch–Tononi image test. 

However, these findings have been methodologically contested: The validity of the control condition as means to rule out partial-awareness effects has recently been put into question (Stein, Hebart, & Sterzer, 2011). The control and experimental conditions in CFS differ profoundly both in observers' subjective experience of the stimuli and in their reaction-time distributions, with greater variability in the experimental condition. When these distributions were matched by mixing experimental and control trials (rather than using a block design typical of previous studies; see Jiang et al., 2007; Mudrik, Breska, et al., 2011; Yang, Zald, & Blake, 2007), the alleged “unconscious” effect was found in both the control and experimental conditions. This may cast doubt on the interpretation that incongruency was unconsciously processed, without any involvement of partial awareness. Furthermore, recent experiments in our own lab (Mudrik, Gelbard-Sagiv, Faivre, & Koch, 2013) have showed that CFS actually involves long periods of partial awareness that are commonly not controlled for, in which subjects can perceive some, but not all, features of the suppressed stimulus (for another demonstration of partial awareness of color during CFS, see Hong & Blake, 2009). Critically, we found that indirect measures of semantic processing (i.e., adaption/priming to famous and nonfamous faces) show effects only during partial awareness and not during complete unconscious processing, where subjects have no access to any feature of the suppressed stimulus. As such partial awareness usually goes unnoticed in many CFS experiments, this implies that some of the reported “unconscious” processing during CFS might actually reflect partial-awareness effects. These findings highlight the methodological shortcomings of CFS, and especially the breaking CFS measure, as a means to evaluate the depth of pure unconscious processing. They therefore call for a reexamination of unconscious integration, especially given the discrepancy between the findings of Mudrik, Breska, and colleagues (2011) and the existing theories of consciousness. 

In the current study, we employed a classical masking paradigm (for reviews, see Breitmeyer & Ogmen, 2000, 2006; Kahneman, 1968) in order to prevent conscious perception of congruent and incongruent scenes. In masking, the visibility of one stimulus (the prime) is reduced by a spatially overlapping previous or subsequent stimulus (mask). Here, a target stimulus (a scene different from the prime that also includes a congruent or incongruent object) followed the prime, in order to indirectly assess the level of unconscious congruency processing: If such processing is possible, we expected to (a) find shorter reaction times for trials in which prime and target belong to the same category (i.e., priming; see, e.g., Cheesman & Merikle, 1984; Dehaene et al., 2001; Draine & Greenwald, 1998) and/or (b) find prolonged reaction times for targets following incongruent primes, as the latter are known to slow down subjects' performance (see, e.g., Bar & Ullman, 1996; Biederman, 1972; Boyce & Pollatsek, 1992). 

This design allowed us to first evaluate the validity of Mudrik, Breska, and colleagues' (2011) findings and their generalizability to other paradigms. Second, it enabled us to test the required exposure duration for unconscious incongruency processing: While in the earlier CFS study the images were presented at full contrast for about 2.5 s (until their emergence into awareness), here they were presented for 33 ms only (and at reduced contrast). Previous studies have found congruency processing of scenes presented for less than 100 ms (Davenport & Potter, 2004) and as short as 26 ms (Joubert, Fize, Rousselet, & Fabre-Thorpe, 2008), but there the scenes were highly visible. Thus, finding evidence for incongruency processing of masked scenes presented for a mere 33 ms would strengthen the claim that some integration of scene elements can indeed be performed in the absence of conscious awareness, even at impoverished presentation conditions. 

Method

Participants

Fourteen healthy volunteers (four women, 10 men; 11 right-handed, three left-handed), aged 19–31 (mean = 23.3), with reportedly normal or corrected-to-normal sight and no psychiatric or neurological history participated in the study for payment (approximately $15 per hour). The experiment was approved by the Institutional Review Board committee of the California Institute of Technology, and informed consent was obtained after the experimental procedures were explained to the subjects. 

Stimuli

Stimuli were 144 pairs of colored real-life scenes that depict a person performing an action with an object (a modification of the stimuli used in Mudrik, Deouell, & Lamy, 2011; Mudrik, Lamy, & Deouell, 2010). In both congruent and incongruent images, the critical object was pasted from another image using Adobe Photoshop software, so that it was either congruent or incongruent with the background (e.g., a baseball player batting with a bat or with a flower bouquet, a woman applying a lipstick or a pickle; see Figure 1). This procedure assures that image artifacts that might occur during the object-insertion procedure occur for both types of scenes. All images were correctly rated by 24 subjects as either congruent or incongruent in a pretest experiment (see Mudrik et al., 2010). The pictures' luminance and contrast levels were digitally equated using the Adobe Photoshop software, and low-level differences in saliency, chromaticity, and spatial frequency were tested using an objective perceptual model (Neumann & Gegenfurtner, 2006). No such differences were found (for details, see again Mudrik et al., 2010), implying that at the group level, the scenes differ in the semantic congruency of the object with its background rather than in the low-level features tested here. Seventy-two of the pairs served as primes and 72 served as targets, so there was no repetition between primes and targets throughout the experiment. Masks were made by dividing the images into a 5×6 matrix and then randomly shuffling the cells (see Figure 1 for examples). Since in the calibration and visibility posttest conditions the task pertained to primes' direction, half the primes and half the masks were presented upside down, and half upright. 

Figure 1

 

(a) Experimental paradigm and stimuli: Two forward masks preceded a congruent or incongruent prime, which was then followed by three backward masks. To achieve maximum masking of the prime scene, the first backward mask was introduced together with six colorful squares that overlapped its edges and extended outside its boundaries. The prime appeared for 33 ms, each mask lasted 50 ms, and all stimuli were followed by a 17-ms blank. This sequence was repeated three times and followed by a target that appeared for another 33 ms and a 17-ms blank. Then questions appeared; in the experimental condition, subjects first had to classify the target as weird or normal, then rate prime visibility on a scale of 1 to 4, and finally classify the prime as weird or normal. In the visibility posttest condition, subjects first classified the prime and then rated its visibility. On the right are enlarged examples of the congruent and incongruent versions of the prime (baseball player batting either with a bat or with a flower bouquet), and target (woman applying lipstick or a pickle). (b) More examples of the congruent and incongruent scenes. From left to right: a girl drinking either from a football or from a water bottle, a boy playing with a drum or with a ball in the water, a man shaving either with a fork or with a razor, a man holding either an eggplant or a flashlight.

Apparatus

Subjects sat in a dimly lit room. The stimuli were presented on a 19-in. CRT monitor with a 60-Hz refresh rate and 1024×768 resolution, using Matlab and Psychtoolbox 3 (Brainard, 1997; Pelli, 1997). They appeared on a gray background at the center of the computer screen and subtended 6.51° (width) × 9.07° (height) of visual angle. Participants' heads were stabilized using a chin rest located 57 cm from the screen. 

Procedure

The experiment included three conditions: calibration, masking, and visibility posttest. In all conditions, the same stimulus sequence was presented: First, two forward masks were presented for 50 ms each, followed by 17-ms blanks (aimed at prolonging stimulus onset asynchrony [SOA] and accordingly enhancing priming; see Vorberg, Mattler, Heinecke, Schmidt, & Schwarzbach, 2003). Then the prime was presented for 33 ms, followed by a blank (17 ms) and three backward masks (50-ms and 17-ms blanks; sandwich masking). To strengthen masking power, the first backward mask was accompanied by six color squares (two blue, two green, and two red), overlapping its borders and extending outwards (Figure 1). This entire sequence was presented three times (Macknik & Livingstone, 1998) and was then followed by a target that appeared at 0.5 contrast (defined by image's transparency, using Matlab's alpha function) for 33 ms in order to make it difficult to identify and possibly facilitate priming, followed by a blank (17 ms). The target was not masked; thus, it was always visible, although subjects did not always manage to fully grasp its content. After the last 17-ms blank, a series of questions was presented, which varied with condition (see following). 

The calibration condition included 72 trials in which half the prime images—congruent and incongruent—were upright and half were inverted (psuedorandomly intermixed, with the constraint that the same prime orientation was never presented in four consecutive trials). Subjects' task was to report the orientation of the prime image while ignoring the target, using the up and down arrow keys, and then to subjectively rate its visibility on the Perceptual Awareness Scale (Ramsøy & Overgaard, 2004), where 1 signifies “I didn't see anything,” 2 stands for “I had a vague perception of something,” 3 represents “I saw a clear part of the image,” and 4 is “I saw the entire image clearly.” In case subjects did not manage to see the prime, they were instructed to guess. To prevent any exposure to the scenes that would be presented as primes in the experimental condition, all images (primes and targets) in the calibration condition were taken from the target-stimuli bank, so they were never presented as primes throughout the experiment. 

Using a staircase procedure (Levitt, 1971), prime and masks contrast were determined for each subject, so as to reach the highest contrast of the prime for which subjects were still at chance performance: Initial masks contrast was 0.85, and initial prime contrast was 0.7. Following correct responses (i.e., correct classification of the image direction as upright or inverted), masks contrast was ramped up by 0.05, and following incorrect responses, it was ramped down by 0.05. If masks contrast reached 1, prime contrast was reduced using the same criterion and the same contrast units (0.05), with a low boundary of 0.4. When subjects completed the calibration, masks contrast was set to the second highest level reached throughout the session—or, if masks contrasts reached 1, it was set to 1—and prime contrast was set to the second lowest level reached throughout the session. Across subjects, averaged masks contrast was 0.94 (SD = 0.08) and averaged prime contrast was 0.62 (SD = 0.08). 

The main experimental condition included 144 trials. Here, all primes were upright and included either a congruent or an incongruent object (primes' congruency was intermixed with the same constraint). Subjects' task was first to report the target's congruency as fast as they could, by pressing either the right or the left arrow key (Q1; see Figure 1; key assignment was randomized between subjects), then to subjectively rate the prime's visibility by pressing the 1 through 4 keys (Q2), and finally to report the prime's congruency using the same keys (objective measure; Q3). Subjects were again instructed to guess if they did not know the answer to either the first or the third question. Every prime image in a pair appeared once, so that throughout the experiment, subjects were exposed to both the congruent and the incongruent versions of each scene exactly once. To counterbalance the order of presentation, the session was divided into two halves, so that within each half, 50% of the images were congruent and 50% incongruent, with no pair repetition. In the second half, every image that appeared as congruent in the first half appeared now as incongruent, and vice versa. Image order within each half was randomized, so subjects could not form any expectation about the order of the images in the second half following the first. 

Finally, the visibility posttest was aimed at achieving a better assessment of stimulus visibility during the main experimental condition. In the latter, the objective measure pertained to the same dimension that was being indirectly measured for the target (scene congruency), to avoid a mismatch between prime and target response (Schmidt & Vorberg, 2006). Yet such a high-level objective measure might not be sensitive enough to lower level awareness of some other aspects of the stimulus that might drive the reported effect (Kouider & Dupoux, 2004; Mudrik et al., 2013). For example, subjects may not be able to perceive enough of the image to report its congruency (leading to chance performance in the objective measure) but still see enough to report a lower level feature, like direction (upright or inverted) or the person's gender. Thus in the visibility posttest, we again presented the same primes that were used in the main experimental condition, but with half upright and half inverted, and asked subjects to report image orientation and visibility. Note that the objective measure in this condition is much stricter than in the main experimental one, as (a) a lower level feature is tested; (b) the critical question appears just following the experimental sequence rather than after two other questions, excluding the possibility that subjects knew the answer but were unable to keep it in memory; and (c) subjects were already well trained with the image and the experimental procedure, so their chances to perform better were higher. 

Results

Overall target and prime classification

Subjects could classify the briefly flashed targets as congruent or incongruent, but only poorly (Q1, see again Figure 1; M = 58%, SD = 4%; t(13) = 7.35, p < 0.001; mean d′ = 0.46, SD = 0.24; t(13) = 7.23, p < 0.001), while they failed to correctly classify the masked primes (Q2; M = 50%, SD = 6%, t(13) = 0.02, p = 0.98; mean d′ = 0.02, SD = 0.30; t(13) = 0.27, p = 0.79). Subjects' poor performance for targets can probably be explained by the very short presentation duration (33 ms) and the reduced contrast (0.5). This suggestion is supported by the apparent difference in classifying congruent and incongruent targets: Subjects seemed to be better at classifying the former (M = 68%, SD = 7%) than the latter (M = 49%, SD = 6%; t(13) = 7.46; p < 0.00001 for the difference between congruent and incongruent targets), though no difference in response time (RT) was found (M = 1.81 s, SD = 0.58 for congruent scenes; M = 1.83 s, SD = 0.66 for incongruent scenes; t(13) = 0.59, p = 0.57). However, this seeming advantage for congruent targets can be explained by subjects' response bias to classify targets as congruent (mean lnβ = 0.12, SD = 0.08; t(13) = 5.44, p < 0.001) when they were unable to identify any “weird” object in the image. Since targets were briefly flashed, subjects often failed to detect such an object, leading to a more frequent classification of the target as congruent. 

Prime visibility

Subjective measure

Subjects' visibility ratings (Q2; Figure 1) confirmed that the masking procedure for the prime was effective: Seventy-four percent of the trials were rated as “I saw nothing” (1), 20% as “I had a vague perception of something” (2), and only 6% as either “I saw a clear part of the image” (3) or “I saw the entire image clearly” (4; Figure 2). Visibility ratings for congruent (75%, 19%, 1%, and 5% for visibility ratings of 1 through 4, respectively) and incongruent (73%, 20%, 1%, and 5%) primes did not differ, as revealed by the lack of interaction in a two-way ANOVA analysis between Congruency (congruent/incongruent) and Visibility rating (saw nothing/vague perception/saw clear part/saw entire image), F(3,39) = 1.43, p = 0.249. No individual image was consistently seen or never seen across all subjects. In all the following analyses, we only report on the 74% of trials in which the prime was not seen at all. 

Figure 2

 

Subjects' visibility ratings for the prime in the experimental conditions (Q2 in Figure 1). Note that about 94% of the trials were rated as either “I saw nothing” or “I had a only vague perception of something.” All analyses of the main experiment were carried out on the nonperceived trials that were classified as “see nothing.”

Objective measure (visibility posttest)

Although subjects were completely at chance in reporting primes' congruency (see overall prime performance previously), the visibility posttest showed that they were significantly above chance for judging the orientation of the masked primes (M = 62%, SD = 7%; t(13) = 6.52, p < 0.0001; mean d′ = 0.62, SD = 0.14, t(13) = 6.59, p < 0.0001). No difference in performance was found between congruent and incongruent primes (M = 63%, SD = 13% for congruent primes; M = 61%, SD = 14% for incongruent primes; t(13) = 0.32, p = 0.76). Importantly, however, subjects' subjective ratings also seem to have changed between the experimental condition and the visibility posttest condition: In the latter, only 51% of the trials were classified as “saw nothing,” 32% as “vague perception,” and 17% as either “saw clear part” or “saw entire image.” Indeed, a two-way ANOVA with Condition (experimental/visibility posttest) and Visibility rating (saw nothing/vague perception/saw clear part/saw entire image) showed a main effect for Visibility rating, F(3,39) = 19.03, p < 0.0001, and critically also an interaction between Condition and Visibility rating, F(3,39) = 5.15, p < 0.005, so that subjects' ratings were different in the visibility posttest as compared with the experimental condition. 

Since the stimulus sequence was identical between the two conditions, this difference could have stemmed from the different tasks, that might have evoked different classification criteria, from the different attentional demands (since in the posttest subjects did not have to attend to the target), or from subjects' higher resilience to the masking due to a generalized training effect (as each image was only used once as a prime). Thus, we computed subjects' accuracy for prime direction using only the 51% of “Saw nothing” trials. There, prime accuracy dropped to chance, as revealed by a one-sampled t test against 0.5 (M = 53%, SD = 11%; t(13) = 0.91, p = 0.38; mean d′ = 0.04, SD = 0.24, t(13) = 0.72, p = 0.48). 

Reaction times

Subjects' RTs for the target (Q1; correct trials only) were analyzed using a two-way ANOVA. Trials with a reaction time outside of three standard deviations were excluded. In order to conduct the ANOVA analysis on the RTs, we first tested whether they are normally distributed. A Shapiro–Wilk normality test conducted on the combined distribution of congruent and incongruent trials showed that the distribution violates the normality assumption (W = 0.90, p < 0.05). Thus, we ran a logarithmic transformation on all the RT values to obtain a normal distribution (W = 0.95, p = 0.19). All subsequent RT analyses were conducted on the logarithmic distribution. In the following, we label all cases in which both prime and target were either both congruent or both incongruent as “same,” and cases in which these two differ as “different.” 

ANOVA (Figure 3) was run with Repetition (same/different) and Prime congruency (congruent/incongruent) as regressors. This revealed a main effect for congruency, F(1,13) = 33.91, p < 0.0001, showing that RTs for targets following an incongruent prime are longer (M = 1.84 s, SD = 0.64 for same; M = 1.90 s, SD = 0.65 for different; nonlogarithmic values) than following a congruent prime (M = 1.73 s, SD = 0.56; M = 1.73 s, SD = 0.52)—irrespective of prime–target relations (hence also irrespective of the target's congruency). At the individual-subjects level, 11 out of 14 subjects showed the effect (binomial test, p = 0.006; Figure 4). We did not find either a main effect for repetition or an interaction between repetition and congruency. Notably, prolonged RTs following incongruent primes were found also when using the raw, nonlogarithmic data, F(1,13) = 16.96, p < 0.002. 

Figure 3

 

A table with mean values of positive and negative standard deviation for both reaction times (left) and accuracy of the subjects in answering Q1 (“target weird”; right) in the four conditions, and p values of statistical comparisons based on the ANOVA analysis (blue; vertical arrows depict a main effect, diagonal arrows an interaction) and subsequent post hoc contrasts (red arrows).

Figure 4

 

RTs for targets with congruent and incongruent primes, for individual subjects, and for invisible trials only. Each purple circle is a subject, with gray lines representing standard errors for that subject. Circles above the diagonal represent longer RTs following an incongruent prime, below the diagonal represent longer RTs following a congruent prime, and on the diagonal represent equal RTs following both types of primes.

Arguably, the congruency effect we found might have stemmed from subjects' familiarity with the scenes in the second half of the experiment: Since every scene that appeared as congruent in the first half appeared as incongruent in the second half, and vice versa, subjects could have formed expectations about scenes' congruency in the second half. If that was indeed the case, one could have expected to find (a) better performance for primes or (b) stronger RT effects in the second half of the experiment. Accordingly, we divided the trials into two halves and examined prime performance and the RT effect in both. No evidence for this possible confound were found; prime performance was at chance for both halves (M = 52%, SD = 9% for the second half; t(13) = 0.75, p = 0.466; M = 48%, SD = 7% for the second half; t(13) = 0.78, p = 0.450). The congruency main effect was in fact only found for the first half, implying that when subjects saw the scenes for the second time—albeit with other objects—they were less affected by their incongruency (F(1,13) = 15.25, p = 0.002, and F(1,13) = 2.13, p = 0.168). Given the low number of trials in each cell in this analysis (18 or less), we cannot determine if this null result reflects lack of effect or a too-low signal-to-noise ratio. 

Accuracy

A two-way ANOVA (Figure 3) run on the accuracy of Q1 (“target weird”) with Repetition (same different) and Prime congruency (congruent/incongruent) as regressors revealed an interaction between repetition and congruency, F(1,13) = 40.58, p < 0.0001, and no main effects. Yet post hoc contrasts showed that this interaction stems from targets' congruency rather than primes': No difference in accuracy was found between congruent and incongruent primes when the target was either congruent, t(13) = 0.18, p = 0.86, or incongruent, t(13) = 0.26, p = 0.80. On the contrary, in the “same” trials, accuracy for congruent primes (hence, congruent targets; M = 66%, SD = 11%) was higher than for incongruent primes (hence, incongruent targets; M = 48%, SD = 12%; t(13) = 4.55, p < 0.001). Similarly, in the “different” trials, accuracy for congruent primes (hence, incongruent targets; M = 49%, SD = 7%) was lower than for incongruent primes (hence, congruent targets; M = 67%, SD = 12%; t(13) = 4.51, p < 0.001). Averaged across all trials, subjects correctly answered Q1 about two thirds of the time (M = 66%, SD = 7%; p < 0.0001) when the target was congruent but were at chance (M = 49%, SD = 6%; p = 0.54) when the target was incongruent. This may reflect subjects' bias to classify targets as congruent, leading to high accuracy for congruent targets and low accuracy for incongruent targets. To overcome this bias and look for potential accuracy difference irrespective of targets' congruency, we calculated d′ in “same” (mean d′ = 0.38, SD = 0.47) and “different” trials (mean d′ = 0.44, SD = 0.44). Still, no difference was found, t(13) = 0.44, p = 0.67. 

Discussion

Our findings show that processing of images that were preceded by invisible images depicting a person performing an appropriate action (e.g., batting at a ball with a baseball bat) is on average 100 ms (SD = 13 ms) faster than processing of images that were preceded by invisible images depicting an inappropriate action (e.g., using a flower bouquet to bat a ball). The images were rendered invisible by short exposure, low contrast, and sandwich masking. Using both subjective and objective measures assured us that subjects were indeed unaware of the masked primes. Notably, our objective test not only controlled for awareness of primes' congruency but also ruled out the option of lower level awareness of more basic prime features such as orientation of the image (upside down or not). The results are not likely to be explained by some systematic low-level difference between congruent and incongruent primes, as these were strictly controlled for (contrast, visual saliency, chromaticity, spatial frequency; see Method). 

The RT difference we found is the first demonstration of ongoing contextual influences of unseen stimuli on subsequent targets. These contextual relations were unconsciously processed here in stimuli of reduced contrast and a very short stimulus exposure (33 ms). Thus, our findings go beyond those of Mudrik, Breska, and colleagues (2011) by, first, overcoming their methodological limitations and, second, extending their findings to show subsequent influence on performance under highly impoverished presentation conditions, in a completely different paradigm. Our findings can further be taken as strengthening previous findings of high-level integrative processes in the absence of awareness (García-Orza et al., 2009; Reber & Henke, 2012; Ric & Muller, 2012; Sklar et al., 2012; Van Opstal et al., 2011). 

Integration in the absence of awareness

Our findings show that some integration can, indeed, take place during unconscious processing. The exact nature of this process remains open. One possibility is that our results demonstrate high-level scene integration and comprehension, where the semantic relations between the object and its background are processed (see, e.g., Boly et al., 2013; Bor & Seth, 2012; Lau & Rosenthal, 2011). An alternative interpretation is that the unconscious processing reflects object-to-object associations, formed by the coactivation of groups of object-selective neurons (see, e.g., Keysers, Xiao, Földiák, & Perrett, 2001; Li & DiCarlo, 2008; Quian-Quiroga, Fried, & Koch, 2013): Upon scene presentation, such neurons encode the different objects that appear in the scene in parallel. In previous exposures to similar scenes where congruent objects tended to appear jointly, populations of object-selective neurons would have already fired together, reinforcing their interconnections and facilitating the processing of congruent scenes. On the contrary, in incongruent scenes the coactivation of selective neurons that rarely fire simultaneously gives rise to conflicts which in turn can lead to delayed visual processing of that scene (Fabre-Thorpe, 2011; note, however, that in our study incongruent scenes hindered the processing of subsequent stimuli, and that in about a third of our images, the incongruency of the object could not have been inferred from its relations with other objects in the scene but only from the way it was held by the person performing the action). Both interpretations, however, imply the occurrence of integration processes: In order for the described conflicts to arise, an interaction between the object-selective neurons should occur. 

If so, how can our findings be reconciled with the common assumption that consciousness involves integration of information (see, e.g., Baars, 2005; Dehaene & Naccache, 2001; Lamme, 2006; Lamme & Roelfsema, 2000; Tononi & Edelman, 1998; Tononi & Koch, 2008)? 

It may be that the claim that consciousness of an image is necessary for any type of object integration to occur needs to be revised. When denied the option of conscious processing, observers seem to nevertheless be able to process the likelihood of an object's appearing in a specific scene. Whether this is achieved via high-level visual-scene integration or through object-to-object integration, it challenges the claim that consciousness is necessary for integrative processes to occur. However, this challenge does not imply that when a viewer is able to consciously perceive a scene, integration is still performed by unconscious processes (much like one can move in space without legs, although we would not want to claim that the function of legs is not enabling movement in space; Koch, 2012; Lau, 2009). Rather, consciousness may regularly operate as a key facilitator of integration, which in turn can occur also during unconscious processing—at least to some extent. 

Notably, our findings do not show that the integrative processes of incongruent scenes were indeed completed without awareness (note that no evidence for congruency priming was found, as discussed later); impaired performance following incongruent scenes could have stemmed from the failure to unconsciously integrate them. Arguably, when integration involves previously learned associations, acquired during past conscious experiences, it can be unconsciously performed (Dehaene et al., 2001; Dehaene & Changeux, 2011). Yet when the scene involves objects that were not previously integrated during conscious perception (i.e., incongruent scenes), integration fails. This failure may subsequently lead to the allocation of additional attentional resources and may thereby hinder subsequent performance (see further discussion later). 

Contextual engagement of attention

The prolonged RTs for targets following incongruent primes are in line with a large literature showing poorer performance for incongruent than congruent stimuli (see, e.g., Bar & Ullman, 1996; Biederman, Glass, & Stacy, 1973; Chun & Jiang, 1998; Kosslyn, 1994; Palmer, 1975; Underwood, 2005). Yet these studies have typically focused on the processing of the congruent or incongruent stimulus itself and explained the performance decline by this stimulus's being harder to process or comprehend. 

Here we show that the processing of not only the actual incongruent stimulus is hindered but also that of a subsequent stimulus presented about 200 ms later. As we suggested earlier, a possible mechanism might be attentional engagement by the incongruent stimulus. Support for this interpretation can be drawn from the results of a binocular-rivalry (BR) study (Mudrik, Deouell, et al., 2011), in which congruent and incongruent objects embedded in the same scenes rivaled against each other. Through differentiating between length and number of dominance periods of the congruent and incongruent objects, that study showed that while attention is not captured by incongruent objects (i.e., the number of incongruent dominance periods was not higher than that of congruent ones), it is nevertheless engaged by it (i.e., once an incongruent object became dominant, it remained so for longer periods; see also De Graef, Christiaens, & Dydewalle, 1990; Underwood, Templeman, Lamming, & Foulsham, 2008). 

An important difference between the current study and the BR one is that here attention seems to have been engaged by incongruent stimuli while they were unconsciously perceived, rather than after they gained dominance during rivalry. Taken together with previous reports of attentional engagement or shifts in the absence of awareness (Ansorge & Heumann, 2006; Jiang, Costello, Fang, Huang, & He, 2006; McCormick, 1997), this can lend support for the notion that attention and consciousness are two distinct processes that sometimes operate independently from one another (Dehaene, Changeux, Naccache, Sackur, & Sergent, 2006; Hardcastle, 1997; Koch, 2004; Koch & Tsuchiya, 2007; Lamme, 2003; van Boxtel, Tsuchiya, & Koch, 2009). 

Lack of congruency priming and accuracy effects

In this study we failed to find any evidence for congruency priming: No RT or accuracy differences were found between trials in which prime and target were of the same category (congruent or incongruent) and trials in which they were of different categories. Two possible explanations could account for this lack of evidence. First, the relatively long SOA between target and prime (200 ms) might have not allowed for priming, which is a short-lived, rapidly decaying phenomenon (Greenwald, Draine, & Abrams, 1996; Mattler, 2005), especially for high-level processes. Alternatively, this could be taken as evidence that unconscious integration is only partial as compared with conscious integration; incongruency may lead to impaired performance on a future target, even without establishing an actual labeling of congruency or incongruency. By this account, the prolonged RTs we found only reflect the failure to form a representation for the incongruent object, which impeded subsequent performance. Future studies using shorter SOAs and examining conscious priming of congruency could shed more light on this issue. 

Furthermore, the effect of incongruent primes on subsequent processing was only reflected in response latency, not in accuracy. This could be explained in two ways. On a more general level, RT measures are considered more sensitive than accuracy ones (see, e.g., Fernandez-Duque & Thornton, 2000; Scherag, Demuth, Rösler, Neville, & Röder, 2004) though for some purposes, like differentiating between healthy and brain-damaged populations, accuracy may be more informative (see, e.g., Swick & Jovanovic, 2002), which accordingly could lead to an effect being found only in the latter measure. Specific to our data, the accuracy measure was probably rendered even less sensitive given subjects' overall poor performance and their relatively strong response bias to classify the targets as congruent. We suggest that subjects tended to classify targets as incongruent only when they managed to detect an incongruent object. Since the targets were briefly flashed, in many trials subjects were unable to do so, leading to a much higher rate of “congruent” responses. This probably overcame any unconscious influence of the primes. 

Conclusions

Our findings provide support for the notion that at least some integration of the constituents of real-life, rich and detailed scenes can occur in the absence of conscious awareness of these scenes. This integration was shown to influence future processing of a subsequent image, possibly due to attentional engagement by incongruent scenes. This study further demonstrates that such unconscious processing can be obtained even when the critical stimulus is presented briefly, with reduced contrast. This suggests that while consciousness might indeed be highly involved in information integration, it may not be necessary for all such integration to occur. 

Acknowledgments

This research was supported by the Human Frontier Science Program, the Weizmann Institute of Science—National Postdoctoral Award Program for Advancing Women in Science, and the G. Harold & Leila Y. Mathers Charitable Foundation. We thank Nathan Faivre, Hagar Gelbard-Sagiv, Uri Maoz, and Julien Dubois for their thoughtful comments and suggestions. 

Commercial relationships: none. 

Corresponding author: Liad Mudrik. 

Email: [e-mail]. 

Address: Division of Biology, California Institute of Technology, Pasadena, CA, USA.