The linear impact of concurrent working memory load on dynamics of Necker cube perceptual reversals

Abstract

Abstract

Abstract:

Abstract  Ambiguous figures are visual stimuli that may be perceived in multistable interpretations. The role of attention in modulating perceptual reversals of ambiguous stimuli is not clear. We tested whether perceptual reversals depend on working memory by manipulating its load while the participants were viewing the Necker cube. Increasing working memory load delayed the latency and decreased the frequency of reversals. These effects followed a linear function of load. The findings imply shared resources of the mechanisms responsible for perceptual reversals and working memory maintenance. However, reversals were not completely abolished even with the hard seven-consonants load, suggesting that bottom-up processes continue to operate in the bistable perception dynamics when top-down mechanisms are attenuated.

Introduction

To effectively act in our environment, we need to use our perceptual system for disambiguation and interpretation of the numerous cues in a given visual scene. The human visual system groups visual features to ensure perceptual organization and stability. Ambiguous figures provide striking examples of how our visual system is striving for stability. The Necker cube is an image that the human visual system perceives as constantly interchanging between two available interpretations, while the image is physically stable. Explanations proposed for the perceptual changes of such figures tend to emphasize the operation of either bottom-up, or top-down perceptual processes (for a review, see Long & Toppino, 2004). The satiation (neural adaptation) theory suggests that perceptual reversals occur via cycles of adaptation, recovery, and mutual inhibition in competing neural channels in early visual areas (e.g., Toppino & Long, 1987). According to alternative cognitive theories (e.g., Leopold & Logothetis, 1999) higher order top-down perceptual processes may be the main cause of reversals. 

In the present study we aimed to investigate how perceptual decision making is affected by the recruitment of attentional resources. Thus we chose a secondary task that is known to deplete the available attentional resources—a working memory load task (Kumar, Soto, & Humphreys, 2009; Singhal & Fowler, 2004). We focused on the possible effects of working memory load on perceptual reversals of the Necker cube. We were motivated by the observations that performing the secondary tasks that require working memory capacity (e.g., mental arithmetic) increases the time for the report of an alternative ambiguous figure percept (Reisberg & O'Shaughnessy, 1984; Reisberg, 1983; Wallace, 1986) and decreases the rate of perceived reversals (Reisberg & O'Shaughnessy, 1984; Wallace & Priebe, 1985; Wallace, 1986). Furthermore, when the participants are actively engaged in a working memory task, they take more time to report the reversal of the ambiguous figure (Reisberg, 1983). Paffen, Alais, and Verstraten (2006) found that concurrent attentional load (i.e., motion-detection task that manipulated working memory and attentional resources) reduces the rate of perceived binocular rivalry alternations. The motion-detection task demands did not degrade the participants' ability to track perceptual changes of rivalrous percepts, as the participants were capable to correctly track pseudorivalry alternations while performing it. However, none of the reported manipulations completely eliminate reversals, implying that perceptual ambiguity involves more than allocation of attentional resources. 

Previous studies using ambiguous stimuli did not manipulate the level of the load, leaving open the possibility that the reductions of perceptual reversals were due to the requirement to perform two tasks simultaneously. Therefore, we selectively chose to include several levels of working memory load manipulation. We studied the influence of working memory on perceptual reversals of the ambiguous Necker cube by manipulating only the concurrent working memory load. The participants kept varying lengths of consonant strings in their memory, while reporting the perceptual reversals of the Necker cube. We assume that the difference between the condition without load and the condition with easiest load will fulfill the dual-task requirements, while the difference between conditions involving different levels of load will present the effect of the working memory load itself. We hypothesize that if loading of working memory reduced the amount of the perceived reversals, this would indicate that attentional resources represent a factor operating in perceptual bistability. 

Methods

Participants

A total of 19 participants (11 males; mean age = 26.7 years, SD = 3.15) took part in the present study. Eighteen participants were right-handed (Oldfield, 1971). Each participant had normal or corrected-to-normal visual acuity and was completely naïve to the hypotheses and goals of the study. All participants gave written informed consent (in accordance to the Declaration of Helsinki) and the institutional ethics committee formally approved the study. 

Stimuli

Memory stimuli

Each trial contained a memory set. It consisted of memory prime and memory probe. The memory prime was either four asterisks (****; sham-load condition), or a string of four to seven capital consonant letters (excluding K, H, and Q; letter-load condition), that is four-letter, five-letter, six-letter, and seven-letter load (4LL, 5LL, 6LL, or 7LL, respectively). The size of an asterisk was 0.4° × 0.4° and the size of each letter about 0.5° × 0.2°. The memory probe consisted of the two arrows (arrow size: 0.5° × 0.8°) to the left (≪) or to the right (≫) in the sham-load condition (Figure 1B), or of one letter in the letter-load conditions (Figure 1C). 

Figure 1

 

(A) An illustration of a single experimental trial: A string consisting of four consonants is followed by the Necker cube and a positive memory probe stimulus. Participants were instructed to memorize the letter stimuli and to press a button each time the cube appeared to reverse. Finally, participants decided if the memory probe was presented (or not) in the initial letter string. (B) Schematic representation of the memory primes and probes in the sham-load condition. (C) Schematic representation of the memory primes and probes in the four-letter-load (4LL), five-letter-load (5LL), six-letter-load (6LL), and seven-letter-load (7LL) conditions. The sizes of the positive probes are enhanced only for illustrative purposes; in the actual experiment all the probes were of identical sizes.

Perceptual stimulus

Ambiguous Necker cube was chosen as experimental stimulus. Cube size was 3° × 3° and it was presented on a 22-inch computer screen with a frame rate of 60 Hz from a viewing distance of 75 cm (Dell, Round Rock, TX; Figure 1A). 

Memory stimuli and the ambiguous Necker cube were presented at the same spatial location in the center of the screen. They were drawn in white (20 cd/m²), presented on a black background (0.4 cd/m²) and viewed binocularly. 

Procedure

Each participant took part in a 60-min individual testing session. Before testing commenced, each participant viewed the Necker cube, and he/she watched it until the participant started perceiving reversals. 

In the beginning of the session, everyone performed 12 practice (four sham-load and eight letter-load) trials to get used to the task requirements. These trials served to familiarize the observers with the timing of stimulus presentations, the importance of fixating on the fixation point, and the operation of the response keyboard. After the practice session, and before the beginning of the test period, participants took a 2-min rest to attenuate any potential fatigue effects from the practice session. 

During the experiment the participants were instructed to view the Necker cube naturally and not to provoke perceptual reversals. They were asked to keep their eyes focused on the central fixation point (size 0.06°), and not to move their eyes within trials. Visual fixation was verified by SIM RED eye tracker with a sample frequency of 500 Hz. The eye tracker was calibrated with a 9-point calibration routine in the beginning of each experimental block. 

On every trial (see Figure 1A), a memory prime was presented for 3 s followed by a 1-s inter-stimulus interval during which a fixation point was presented. Then the Necker cube was presented in the center of the screen for 10 s and participants had to indicate each perceptual change by pressing a designated button on the response keyboard with their index finger. The cube was followed by a blank screen presented for 0.5 s. Finally, the memory probe was presented. It consisted of the arrows to the left or to the right in the sham-load condition, or of one letter that either had been part of the memory set (positive probe) or had not been part of the memory set (negative probe) in the letter-load conditions. Participants discriminated left versus right arrows and positive versus negative probes by pressing the designated response buttons using their index and middle fingers. Positive and negative probes were equiprobable and presented randomly. A 5-s intertrial interval was provided after each trial. 

The experimental session comprised 150 trials in total (30 sham-load trials and 30 trials in each letter-load condition), presented in two separate blocks. Blocks were separated by a 5-min break. 

Statistical analyses

To be sure that the participants paid attention to the working memory load task, we included only perceptual reversals from the trials followed by the correct responses to memory probes in further analyses of the reversal rates and response times to reversal. After exclusion of erroneous working memory load trials, on average (SD) 30 (0.23) trials in the sham-load condition, 28.2 (1.78) in the 4LL, 25.6 (2.34) in the 5LL, 23.9 (2.47) in the 6LL and 23.2 (2.41) in the 7LL conditions remained for analyses of the reversal rates. For each load condition, we divided the number of reversals obtained during correct working memory load trials (and all reversals in the sham-load, respectively) by the amount of correct working memory load trials. These new reversal rate values were calculated for each participant individually and used in the analyses. 

For a participant's data to be included in the analyses of response times to reversal, we set up a criterion that there should be a minimum of 15 trials with at least two perceptual reversal responses in each load condition. Based on this criterion, five participants were excluded from the analyses of response times to reversal, thus on average (SD) 30 (0) trials in the sham-load condition, 28.4 (1.50) in the 4LL, 26.1 (1.83) in the 5LL, 24 (2.39) in the 6LL and 23.9 (2.27) in the 7LL conditions remained for analyses of the response times to reversal. 

Reversal rate, D(19) ≥ 0.10, p > 0.05, probe response time, D(19) ≥ 0.15, p > 0.05, and response time to reversal, D(14) ≥ 0.10, p > 0.05, values were normally distributed (Kolmogorov−Smirnov test). One-way ANOVAs with a within-subject factor of Working Memory Load (sham-load, 4LL, 5LL, 6LL, and 7LL) were performed on the mean values of working memory task (accuracy, probe response time) and reversal rate values. A 2 (Reversal: first reversal, second reversal) × 5 (Working Memory Load: sham-load, 4LL, 5LL, 6LL, and 7LL) ANOVA was conducted on the mean values of response time to reversal. In the case of significant Reversal × Working Memory Load interaction, separate repeated measures ANOVAs with one within-participant factor of Working Memory Load (sham-load, 4LL, 5LL, 6LL, and 7LL) were conducted on the response time to reversal data. Post-hoc t tests (Fisher's Least Significant Difference, hereafter Fisher's LSD) were used for pairwise comparisons of conditions in the case of a main effect of Working Memory Load. The significant linear trends in the data revealed by the within-subjects contrasts were also reported. In the statistical analyses, we reported the original degrees of freedom together with effect sizes (partial eta squared: ηp²), but corrected the p-values according to Huynh−Feldt correction whenever the degrees of freedom were greater than 1. 

Results

Working memory task performance

One-sample t tests

We conducted one-sample t tests on the accuracy rates in the concurrent working memory task to verify that participants were paying attention to the memory primes. The accuracy in all experimental conditions was significantly different from chance performance (i.e., accuracy of 50%): sham-load, t(18) = 270.43, p < 0.0001, 4LL (t[18] = 32.11, p < 0.0001), 5LL (t[18] = 20.09, p < 0.0001), 6LL (t[18] = 15.67, p < 0.0001), and 7LL (t[18] = 14.70, p < 0.0001). 

Accuracy and working memory load

The accuracy in responding to the memory probe decreased with harder working memory load task, F(4, 72) = 53.03, p < 0.0001, ηp² = 0.75, and the decrease was linear, F(1, 18) = 198.82, p < 0.0001, ηp² = 0.92, with augmentation in working memory load (Figure 2A). 

Figure 2

 

(A) The accuracy in responses to the memory probes under sham-load, four-letter-load (4LL), five-letter-load (5LL), six-letter-load (6LL) and seven-letter-load (7LL) conditions (N = 19). Error bars represent ±1 standard error (SEM) above and below the mean. (B) The response times to the probes under sham-load, four-letter-load (4LL), five-letter-load (5LL), six-letter-load (6LL), and seven-letter-load (7LL) conditions (N = 19). Error bars represent ±1 SEM above and below the mean.

The accuracy in the sham-load condition was higher than those acquired in response to all other conditions (all p-values < 0.002, Fisher's LSD post hoc comparisons) and accuracy in response to 4LL was higher than those in response to 5LL, 6LL, and 7LL (all p-values < 0.0001, Fisher's LSD). The accuracy in response to 5LL was higher than those in response to 6LL and 7LL (all p-values < 0.02, Fisher's LSD). There was no significant difference between the probe response accuracies obtained in response to 6LL and 7LL (p = 0.30). 

Probe response times

Probe response times increased with working memory load, F(4, 72) = 53.51, p < 0.0001, ηp² = 0.75 (Figure 2B), and the increase was linear with augmentation in working memory load, F(1, 18) = 64.48, p < 0.0001, ηp² = 0.78. 

The probe response times in sham-load condition were shorter than those obtained in all other conditions (all p-values < 0.0001, Fisher's LSD). The probe response times to 4LL and 5LL were shorter than those in response to 6LL and 7LL (all p-values < 0.02, Fisher's LSD). The 6LL probe response times were shorter than those obtained in response to 7LL (p < 0.03, Fisher's LSD). 

Responses to reversals

Reversal rate

The amount of perceptual reversals decreased as working memory load increased, F(4, 72) = 3.26, p < 0.04, ηp² = 0.15, and the decrease in reported perceptual reversals was linear, F(1, 18) = 6.14, p < 0.03, ηp² = 0.25, with augmentation in working memory load. 

The highest rate of reversals was obtained in response to sham-load trials (Figure 3A), the lowest—in response to 7LL trials. The amounts of perceptual reversals in response to sham-load, 4LL and 5LL were significantly higher than that in response to 7LL (all p-values < 0.03, Fisher's LSD). The amount of perceptual reversals reported under 6LL and 7LL did not differ significantly from each other (p = 0.14). Importantly, the effect of working memory load on reversals was small (from about 2.9 reversals/trial in the sham-load and 4LL to about 2.6 in 7LL). Thus, a considerable number of reversals occurred even during the hardest load. 

Figure 3

 

(A) The average number of reversals per trial under sham-load, four-letter-load (4LL), five-letter-load (5LL), six-letter-load (6LL), and seven-letter-load (7LL) conditions (N = 19). Error bars represent ±1 SEM above and below the mean. (B) The response times to the first and second perceived reversals under sham-load, four-letter-load (4LL), five-letter-load (5LL), six-letter-load (6LL), and seven-letter-load (7LL) conditions (N = 14). Error bars represent ±1 SEM above and below the mean.

Response time to reversal

The 2 (Reversal: first vs. second) × 5 (Working Memory Load) within-subjects ANOVA revealed significant effect for Working Memory Load, F(4, 52) = 3.55, p < 0.03, ηp² = 0.21, indicating that the response times to reversal increased as the load increased. A within-subjects contrast showed a significant, F(1, 13) = 11.55, p < 0.006, ηp² = 0.47, linear trend revealing that the harder was working memory load task, the longer it took for the participants to perceive the reversals of the subsequently presented ambiguous figure (Figure 3B). Significant Reversal × Working Memory Load interaction, F(4, 52) = 6.67, p < 0.0001, ηp² = 0.34, was obtained. Subsequent analyses revealed a significant effect for Working Memory Load only for the response time to the second reversal, F(4, 52) = 5.57, p < 0.003, ηp² = 0.30. This effect was as well linear, F(1, 13) = 11.55, p < 0.006, ηp² = 0.47. The response time to the second reversal obtained under sham-load was significantly shorter than those in responses to 5LL, 6LL, and 7LL (all p-values < 0.04, Fisher's LSD). The response times to reversal under 4LL were significantly shorter than those in responses to 6LL and 7LL (all p-values < 0.007, Fisher's LSD). The response times to reversal under 5LL and 6LL (p = 0.19) or 6LL and 7LL did not differ significantly (p = 0.68). 

Distribution of fixations

Fixations were calculated based on the recorded gaze behavior using BeGaze software (SMI, Teltow, Germany). A fixation duration threshold of 150 ms was used. Fixations that had the same position and were separated by a blink were concatenated. A rectangular area of interest (0.76° × 1.07°) was defined on the center of the Necker cube. We excluded fixations falling outside the area of interest from the analyses of fixation distributions. This removed ∼ 3% of the fixations made during the experiment. To calculate dwell time, consecutive fixations were concatenated. The participants successfully maintained their gaze in the area of interest for > 90% of all experimental time. Figure 4 shows the distribution of the fixations of five representative observers during the whole experiment. 

Figure 4

 

The distribution of the fixations of five representative observers during the whole experiment.

Discussion

We studied the role of available attentional resources on perceptual stability by manipulating working memory load while the participants were observing an ambiguous Necker cube. The results revealed that working memory load modulated the dynamics of perceptual stability, but perceptual reversals were not completely abolished even with the hard seven-consonant load. 

The manipulation of the working memory load was effective, as shown by the linear decrease in accuracy and linear increase in probe response times. Moreover, when the participants deployed their attentional resources on the concurrent working memory load task, they perceived less reversals of the ambiguous Necker cube and the latency of perceived reversals increased as a linear function of load. For example, the participants took around 300 ms longer to report the second perceptual reversal under 7LL in comparison to sham-load and 4LL. These findings support the hypothesis that the resources used in maintaining the stimuli in working memory play a definable role in the perception of ambiguous Necker cube. 

When the attention of observers is diverted due to a secondary task, they perceive fewer reversals (Alais, van Boxtel, Parker, & van Ee, 2010; Paffen et al., 2006; Reisberg & O'Shaughnessy, 1984; Reisberg, 1983; Wallace & Priebe, 1985; Wallace, 1986). Reisberg and O'Shaughnessy (1984), Reisberg (1983), Wallace and Priebe (1985), and Wallace (1986) explored the effects of secondary (i.e., mental arithmetic or working memory load) tasks on the perception of ambiguous figures. However, they did not employ a comparable control task to separate out the sole effects of the secondary task, and most importantly, lacked a sufficient number of load levels to reveal a possible linear function. In contrast, we kept the dual-task requirement constant and found that the level of working memory load modulated the rate and latency of perceptual reversals in a linear manner. Therefore, the effects obtained in our study were caused by working memory load and not just by dual-task requirements. 

In principle, the effects of working memory load manipulation on the reversal dynamics could occur due to the encoding of the letter strings: The longer the memory prime, the more time it takes to encode it into the working memory; hence in the early phase of the trials the encoding might have been still going on and thus the reversals were delayed. While this argument may apply to some of the earlier studies that have shown that attention delays the latency of the first reversal (Reisberg, 1983; Wallace, 1986), it is not relevant in the case of the present study. We found that particularly the second reversals were delayed by working memory load (Figure 3B). This finding could not occur due to the prolonged encoding of the letter strings or dual-task demands, which would have influenced only the first reversal, not the second one. Therefore, our results suggest that working memory and perception of ambiguous Necker cube might use the same mechanism(s) or at least are tightly linked in terms of top-down control. These top-down mechanisms may not operate in presentation modes that encourage more bottom-up driven reversals, such as the intermittent presentation modes commonly used in event-related potential (ERP) studies (Kornmeier & Bach, 2004, 2006). Accordingly, we previously manipulated perceptual load (Intaitė, Koivisto, & Revonsuo, 2013) and working memory load (Intaitė, Koivisto, & Castelo-Branco, revised and resubmitted) in ERP studies using intermittent Necker cube presentation, but did not obtain behaviorally observable effects on the frequency of perceptual reversals. 

We did not address the questions of the format of the attentional resources, or the possible effects of working memory load on bistability in other modalities, for example—auditory (Pressnitzer & Hupé, 2006). Auditory bistability shares common features with visual bistability: The percepts are mutually exclusive, their duration fits gamma distribution, and auditory bistability is partially influenced by participants' intentions (Pressnitzer & Hupé, 2006). In the future experiments, it would be valuable to manipulate the format of the attentional resources used concurrently with the task of perceptual bistability to gain better understanding of the mechanisms of bistable perception. Furthermore, extending the duration of the trial could be an interesting experimental manipulation to test how perceptions of reversals would be influenced when participants had to perform working memory load task for longer durations. 

Many studies indicate that both top-down and bottom-up processes are essential in the perception of ambiguous images (Hochberg & Peterson, 1987; Intaitė, Koivisto et al., 2013; Intaitė, Noreika, Šoliūnas, & Falter, 2013; Kornmeier & Bach, 2012; Kornmeier, Hein, & Bach, 2009; Leopold & Logothetis, 1999; Long, Toppino, & Kostenbauder, 1983; Long & Toppino, 2004). Kornmeier and Bach (2012) suggested an integrative theory of bistable perception. They assume that during an observation of the ambiguous figure, the “currently seen” percept gets destabilized in a rather slow manner. After the destabilization, a fast restabilization (disambiguation) takes place, resulting in an alternative percept of the ambiguous figure. According to this theory, both bottom-up and top-down factors can influence the reversal process concurrently. In our study, the working memory load task was presented in the beginning of each trial; it may have established a prioritized top-down tonic control of the circuits involved in perceptual reversals. Therefore, the destabilization effects were modified via similar mechanism(s) as a “top-down intentional” instruction to stabilize the percepts, and hence working memory load decreased reversal rates and increased response times to reversal. 

Conclusion

In summary, our results show a linear effect of working memory load on perceptual stability, suggesting a top-down control mechanism that works under constant (long) presentation modes. However, the influence of load was small and reversals continued to occur under hard seven consonants load. This pattern suggests that shared attentional resources are used for maintaining stimuli in working memory and processing of the ambiguous Necker cube, but even under hard levels of load the bottom-up processes are still able to generate alternate perceptual interpretations of the Necker cube. 

Acknowledgments

This study was supported by the fellowship (SFRH/BPD/77563/2011) to M. I. and PEst-C/SAU/UI3282/2013, CENTRO-07-ST24-FEDER-00205 grants to M.C.B. from Portuguese Foundation of Science and Technology. M. K. was supported by the Academy of Finland (projects 125175 and 218272). We would like to thank two anonymous reviewers for their helpful comments and suggestions. 

Commercial relationships: none. 

Corresponding author: Monika Intaitė. 

Email: [e-mail]. 

Address: Visual Neuroscience Laboratory, IBILI, Faculty of Medicine, University of Coimbra, Coimbra, Portugal.