Combining spatial and temporal expectations to improve visual perception

Abstract

Abstract

Abstract:

Abstract  The importance of temporal expectations in modulating perceptual functions is increasingly recognized. However, the means through which temporal expectations can bias perceptual information processing remains ill understood. Recent theories propose that modulatory effects of temporal expectations rely on the co-existence of other biases based on receptive-field properties, such as spatial location. We tested whether perceptual benefits of temporal expectations in a perceptually demanding psychophysical task depended on the presence of spatial expectations. Foveally presented symbolic arrow cues indicated simultaneously where (location) and when (time) target events were more likely to occur. The direction of the arrow indicated target location (80% validity), while its color (pink or blue) indicated the interval (80% validity) for target appearance. Our results confirmed a strong synergistic interaction between temporal and spatial expectations in enhancing visual discrimination. Temporal expectation significantly boosted the effectiveness of spatial expectation in sharpening perception. However, benefits for temporal expectation disappeared when targets occurred at unattended locations. Our findings suggest that anticipated receptive-field properties of targets provide a natural template upon which temporal expectations can operate in order to help prioritize goal-relevant events from early perceptual stages.

Introduction

Perceiving is a highly active process, guided flexibly and dynamically by proactive predictions about upcoming events. In nature such predictions often combine information about the location (where) and timing (when) of relevant events. In the laboratory, however, we tend to study each type of expectation in isolation (for further discussion see Nobre, 2010; Nobre & Rohenkohl, 2014). We risk, therefore, missing important interactions between different sources of expectations. 

The important role that temporal expectations play in enhancing visual perception is becoming increasingly clear. Electrophysiological studies in nonhuman primates have shown that visual areas such as V1 (Lakatos, Karmos, Mehta, Ulbert, & Schroeder, 2008; Lima, Singer, & Neuenschwander, 2011; Shuler & Bear, 2006), V4 (Ghose & Maunsell, 2002), MT (Ghose & Bearl, 2010), and IT (Anderson & Sheinberg, 2008) are modulated by temporal expectations. These results have been supported by studies in humans showing enhancement of the early visual P1 potential in response to targets with high temporal expectation (Doherty, Rao, Mesulam, & Nobre, 2005; Rohenkohl & Nobre, 2011; Zanto et al., 2011). Additionally, studies in humans and nonhuman primates have shown that temporal expectation modulates anticipatory neural activity to enhance the perceptual processing of events occurring at expected times (Besle et al., 2011; Cravo, Rohenkohl, Wyart, & Nobre, 2013; Green & McDonald, 2010; Lima et al., 2011; Praamstra, Kourtis, Kwok, & Oostenveld, 2006; Praamstra & Pope, 2007; Rohenkohl & Nobre, 2011; Stefanics et al., 2010; van Ede, de Lange, Jensen, & Maris, 2011). 

Recent findings have also demonstrated behavioral consequences linked to perceptual modulation by predictable structure of events (Vangkilde, Petersen, & Bundesen, 2013; for a review see Nobre & Rohenkohl, 2014). For example, temporally predictive symbolic cues enhance perceptual sensitivity (d′) for identifying target letters embedded within a stream of distractors under rapid-serial-visual-presentation (RSVP) conditions (Correa, Lupiáñez, & Tudela, 2005; Davranche, Nazarian, Vidal, & Coull, 2011). Studies manipulating the appearance of visual targets relative to irregular or regular rhythms also report perceptual benefits for targets appearing at moments predicted by the rhythmic structure (Cravo et al., 2013; de Graaf et al., 2013; Mathewson, Fabiani, Gratton, Beck, & Lleras, 2010; Rohenkohl, Cravo, Wyart, & Nobre, 2012). 

Though perceptual effects of temporal expectation by now are undisputed, the means through which temporal expectations can influence perceptual analysis remains largely mysterious. One possibility is that temporal expectation is a form of temporally specific alertness signal (Posner & Boies, 1971; Posner & Petersen, 1990; see also Weinbach & Henik, 2012). Temporal cues could induce a brief up-regulation of neural activity throughout neurons in sensory cortices to amplify processing at the time of the anticipated event (see Nobre, 2010; Nobre, Rohenkohl, & Stokes, 2012). However, this explanation seems inadequate or at least incomplete. This type of mechanism would be metabolically demanding and would lack selectivity to separate a putative target from simultaneous distractors. Furthermore, using pharmacological manipulations, it is possible to dissociate effects of selective temporal expectation from those of general alertness in cueing tasks (Coull, Nobre, & Frith, 2001). A recent, promising proposal is that temporal expectation operates synergistically with other anticipatory top-down biasing mechanisms to enhance their effect (Doherty et al., 2005; Nobre, 2010; Nobre & Rohenkohl, 2014). For example, spatial expectation leads to relative increases in firing rate in neurons with receptive fields covering the attended location (Luck et al., 1997), and leads to relative desynchronisation of low-frequency alpha-band oscillations at the neuronal ensemble level. By timing these other top-down mechanisms, it becomes possible for temporal expectation to magnify their effects. The strengthening of other biases, based on receptive-field properties (such as location, orientation, or motion) is a powerful, natural, and selective means through which temporal expectation can enhance the perceptual analysis of stimuli. 

The synergistic effect between temporal and spatial expectations was originally observed by Doherty et al. (2005). They used rhythmic motion to compare and contrast modulatory effects of temporal and spatial expectations. In their study, a small disc moved in discrete steps across a display containing an occluding band. The disc moved toward the occluding band at a regular or irregular pace, following a linear or unpredictable spatial trajectory. Upon its reappearance, participants had to discriminate whether the disc contained a high contrast small black dot in its center (50% of trials). They made a speeded response in target-present trials and refrained from responded if the disc was empty. Noninvasive event-related potential (ERP) recordings indicated that spatial expectation alone (i.e., under temporal uncertainty) increased the magnitude of visual P1 potentials contralateral to spatially expected targets, in line with spatial orienting effects in spatial cueing tasks (Eimer, 1994; Mangun & Hillyard, 1987). In contrast, isolated temporal expectations had no effect on the first visual potential (see also Correa & Nobre, 2008). Strikingly, however, temporal expectation greatly potentiated the P1 gain modulation by spatial attention. The authors suggested that temporal expectation may influence early stages of perceptual processing by interacting with predictions about spatial receptive-field properties in order to time the excitability of the relevant neuronal pool (see also Anderson & Sheinberg, 2008; Ghose & Bearl, 2010; Ghose & Maunsell, 2002; Lima et al., 2011). 

However, there was no evidence of this synergistic interaction between temporal and spatial expectations at the level of behavioral performance. Accuracy and reaction-time effects of temporal and spatial expectations were independent. The lack of a behavioral interaction is not surprising given the task parameters, and it does not invalidate the synergistic hypothesis. The task used by Doherty and colleagues (2005) was not optimal for revealing the consequences of improved perceptual analysis on behavior (see also Green & McDonald, 2010). The perceptual discrimination required was not demanding and the high accuracy levels may not have left scope to observe fine modulations. In contrast, the later stages of response mapping and selection were demanding, and may have introduced additional large bottlenecks for performance. The critical test of whether synergist interactions between temporal and spatial expectations can optimize perceptual judgments is still missing. 

We address this important outstanding question in the present study. We tested whether interactions between temporal and spatial expectations can have perceptual consequences at the behavioral level. We tested this hypothesis directly by using a factorial design to manipulate independently spatial and temporal expectations about imperative target stimuli. We used a psychophysical task in which perceptual sensitivity functions should impose the primary limits in performance variables. Foveally presented cues were used to manipulate predictions about the location (spatial expectation) and time (temporal expectation) of upcoming targets (Figure 1). The shape of the cue indicated whether the target was more likely to appear on the left or right side of the screen, while its color indicated when the target would appear. This factorial manipulation allowed us to investigate the interaction between spatial and temporal predictions, as well as their effects in isolation. 

Figure 1

 

Schematic illustration of the task structure. Foveally presented cues predicted simultaneously where and when target events were more likely to occur. Cue validity for both spatial and temporal expectation was fixed at 80%. Targets consisted of a horizontally or vertically oriented Gabor patch followed by a backward pattern mask. Targets were presented at a fixed contrast, individually adjusted to equate discrimination performance across individuals. Observers responded to the Gabor orientation with their left of right index finger (counterbalanced across subjects).

Methods

Observers

Twenty right-handed individuals participated in this experiment (mean age = 24.75 years; SD = 5.08 years; 12 females). Visual acuity was normal or corrected-to-normal. All experimental methods had ethical approval from the Central University Research Ethics Committee of the University of Oxford and were conducted in accordance with the Declaration of Helsinki. 

Apparatus

Stimuli were created on MATLAB v.7.10 (The MathWorks, Inc., Natick, MA) and presented using the Psychtoolbox v.3.0 package for MATLAB (Kleiner, Brainard, & Pelli, 2007). The stimuli were displayed on a 22-in. monitor (Samsung SyncMaster 2233, Samsung Electronics, Co. Ltd., Seoul, Korea) with a spatial resolution of 1680 × 1050 pixels and a refresh rate of 60 Hz, placed 76 cm in front of the participant. A chin rest was used to maintain a constant viewing distance and head position. Binocular eye positions were recorded with a desktop mount video-based eye tracker at 500 Hz (EyeLink 1000, SR Research, Ontario, Canada). Responses were collected via a response box (DirectIN High Speed Button; Empirisoft Corp., New York, NY). 

Stimuli and task

A schematic of the display sequence is shown in Figure 1. Observers were asked to discriminate the orientation (horizontal or vertical) of peripheral target stimuli, which were preceded by foveally presented cues indicating where and when these targets were more likely to occur. Trials commenced with the presentation of a fixation dot (diameter: 0.8° of visual angle) and two luminance pedestals positioned at 4.7° below the horizontal meridian and ±3.3° from the vertical meridian (10% contrast). The luminance pedestals were used to reduce spatial uncertainty, indicating the two possible target positions (Gould, Wolfgang, & Smith, 2007; Smith, 2000; Smith, Ratcliff, & Wolfgang, 2004; Smith & Wolfgang, 2004; Smith, Wolfgang, & Sinclair, 2004). Then, after a random interval between 750 and 1200 ms, a cue was presented foveally (2.1° visual angle). Cues consisted of isoluminant colored arrows (line width: 0.1°), presented for 200 ms. The direction of the arrow indicated whether the targets were more likely to appear (left or right pedestal), while its color (pink or blue) indicated when the target would be presented (early: 800 ms or late: 2000 ms; stimulus-onset asynchrony [SOA]). Left and right cues were equiprobable, as were short and long cues. Cue validity for both spatial and temporal conditions was fixed at 80% to encourage use of the cue information. Like in many other temporal orienting experiments, expectations at the two target intervals were asymmetric. The temporal conditional probabilities for targets appearing at the short interval were 0.8 and 0.2 for valid and invalid temporal cues respectively. However, once the short interval has lapsed, the probability of a target being presented at the long interval was always 1.0. Targets were presented atop the luminance pedestals for 50 ms followed by a 283-ms backwards-mask (117-ms target-mask SOA). Backwards interruption masks were used to disrupt early iconic stimulus traces (Smith & Ratcliff, 2009). This prevented participants from retroactively attending to stimuli presented at unexpected times and ensured that behavioral performance in the task was limited by participants' attentional state at the time a target appeared. Target-mask SOAs were chosen based on previous studies showing that backward pattern masking is still effective at ∼100 ms, and by pilot testing (Breitmeyer, 1984; Smith & Ratcliff, 2009; Smithson & Mollon, 2006; Turvey, 1973). Targets consisted of a horizontally or vertically oriented Gabor patch with spatial frequency of two cycles per degree of visual angle. Target contrast was individually adjusted using a staircase procedure (see Procedures below). Backward-mask stimuli were constructed by applying a Gaussian-vignette to the convolution of 100% contrast square-wave gratings at the two possible target orientations. The diameter of the target and mask stimuli was 1.96° of visual angle, and their center was positioned atop the luminance pedestals. All stimuli were presented against a uniform midgray background. Responses to each target were made with the left or right index finger according to the target orientation. The response mapping and cue-color assignment to each cue-target interval were counterbalanced across observers. 

Procedures

Each observer completed a calibration session before the experiment using an adaptive psychophysical staircase procedure to estimate the threshold contrast for perceiving the Gabor gratings (Kaernbach, 1991). Task difficulty was adjusted for each participant by titrating the contrast of the Gabor patch for which orientation discrimination was performed at 75% accuracy. The calibration was performed on a minimum of 120 trials containing valid spatiotemporal cues only. Additional calibration trials were used when the observers' contrast was inconsistent. After calibration was complete, observers completed a practice session containing a minimum of 50 trials. This training session also contained only spatiotemporal valid cues, and was used to ensure that observers formed a strong association between the color of the cues and its respective interval. 

All observers completed the task seated comfortably in a dimly lit room with their heads position stabilized by a chin rest. They were required to maintain central fixation throughout the experiment. They were instructed to use the spatial and temporal information indicated by the cues to predict where and when targets were more likely to occur. The experimental session consisted of 20 blocks of 40 trials each (800 trials total), lasting for approximately 1 hr. The number of trials for short and long intervals by condition was 256 in attended location and time, 64 in attended location and unattended time, 64 in unattended location and attended time, and 16 in unattended location and time. Trials were presented in a randomized order. Rest breaks were provided at the end of each block. 

Data analysis

Statistical analysis was performed using MATLAB and SPSS. Where appropriate, the Greenhouse-Geisser correction for nonsphericity was applied. Saccades were detected using a velocity-based algorithm (Engbert & Mergenthaler, 2006). Trials containing saccades (∼5% trials) were removed from all analyses. 

Behavioral performance was analyzed using perceptual sensitivity values (d′), proportion of correct responses (Pc) and response times. Sensitivity to stimulus orientation was calculated according the formula:  

 

where Pc_H and Pc_V correspond to the observers' proportions of correct responses to horizontal and vertical stimuli respectively, and z to the inverse normal (z score) transformation. 

Response times were adjusted according to the observers' accuracy in order to account for possible speed-accuracy trade-offs. Similarly to previous studies, this adjustment was done using a measure called inverse efficiency (Chambers, Stokes, & Mattingley, 2004; Romei, Driver, Schyns, & Thut, 2011; Townsend & Ashby, 1983). The inverse efficiency (IE) is calculated by dividing the mean reaction time by the proportion of correct responses. 

Repeated-measures ANOVAs were performed using the Greenhouse-Geisser correction for nonsphericity. Additionally, we used nonparametric permutation tests to control for possible biases in the statistics due to difference in power between valid and invalid conditions (Ernst, 2004; Maris, Schoffelen, & Fries, 2007). This procedure consisted of multiple stages (for a more detailed description of the analysis, see Ernst, 2004). First, we calculated the mean difference between conditions of interest for each participant. We then performed F tests (ANOVAs) or t tests (for post-hoc comparisons) on these values. Both statistical tests were two-sided and used a critical alpha level of 0.05. We assessed the significance of the observed results by comparison to a null distribution generated via Monte Carlo simulation with 10,000 repetitions. This null distribution was generated by randomly shuffling the condition labels within each participant's data in each repetition. We then performed the statistical test (F or t tests) on the mean difference between the conditions of interest, and the resulting value was entered into the null distribution. The permutation p value was determined as the proportion of random partitions that resulted in a larger test statistic than the observed one. 

Results

Perceptual sensitivity values (d′) for discriminating targets were initially submitted to an omnibus repeated-measures ANOVA with spatial expectation (expected, unexpected), temporal expectation (expected, unexpected), and interval (short, long) as factors. As explained in the Methods section, effects of temporal expectation should occur primarily or exclusively in short-interval trials. A synergistic interaction between temporal and spatial expectation should therefore result in a three-way interaction—showing significant enhancement of perceptual discrimination of targets appearing at the short interval by temporal expectation when these occurred at the expected spatial location, but not at the unexpected location. The predicted three-way interaction was indeed significant, F(1, 19) = 7.13, p = 0.015; F-test permutation, p = 0.011. Follow-up analyses confirmed there were no effects of temporal expectation in the long-interval trials (see Supplementary Table S1). Accordingly, we focused our analyses on trials in which the target was presented at the short interval (a description of the results in the long condition can be found in the Supplemental Materials). In addition, the omnibus analysis revealed main effects of spatial expectation, F(1, 19) = 29.06, p = 3.34 × 10⁻⁵; F-test permutation, p < 10⁻⁴, with higher performance for targets presented at the expected (M = 1.59, SEM = 0.07) than unexpected (M = 0.84, SEM = 0.07) locations; and of interval, F(1, 19) = 5.03, p = 0.037; F-test permutation, p = 0.033, with improved performance at the short interval overall. 

Figure 2a shows d′ values for targets presented at the short intervals. These values were submitted to repeated-measures ANOVA with spatial and temporal expectations as factors. This analysis again revealed a significant interaction between temporal and spatial expectations, F(1, 19) = 7.10, p = 0.015; F-test permutation, p = 0.018. This was accompanied by a main effect of spatial expectation, F(1, 19) = 23.76, p = 1.05 × 10⁻⁴; F-test permutation, p <10⁻⁴, and no main effect of temporal expectation, F(1, 19) = 0.05, p = 0.829; F-test permutation, p = 0.844. Subsidiary post hoc analyses indicated that temporal expectation improved performance only when targets were presented at the attended location (Figure 3, right histogram; t-test permutation, p = 0.008). When targets were presented at the unattended location, there was no benefit of temporal expectation, but instead a weak trend for a cost (Figure 3, left histogram; t-test permutation, p = 0.172). Figure 3a shows that the key effects—interaction between spatial and temporal expectations, main effect of spatial expectation, and no effect of temporal expectation—were consistent across observers. This pattern of results was equivalent when the outliers shown in Figure 3 were removed, and when the analysis was conducted using an equivalent number of trials for each of the experimental conditions (see Supplementary Table S2). In addition, an equivalent pattern of statistical inferences was obtained for the analysis of accuracy data (proportion of correct trials, see Figure 2b and Supplementary Figure S1). 

Figure 2

 

Behavioral results. Effects of spatial and temporal expectations on (a) d′, (b) accuracy, and (c) inverse efficiency. Error bars represent SEM.

Figure 3

 

Effects of temporal expectations on d′ measures at the individual level and permutation tests. Scatter and bagplots showing the main effects of temporal and spatial expectations (Top) and the effect of temporal expectation at the attended and unattended locations (Bottom). Blue indicates the conditions in which there was a significant effect of expectation. The cross at the center of the bagplot represents the center of mass of the bivariate distribution of empirical data, the darker area includes 50% of the data with the largest depth, the lighter polygon contains all other nonoutlier data points, and the Xs represent outliers (Rousseeuw, Ruts, & Tukey, 1999). Outliers were detected using the “Skewness-Adjusted Outlyingness” method using LIBRA toolbox for MATLAB (Verboven & Hubert, 2005; for more details on this method see Hubert & Van der Veeken, 2008). Histograms showing the distributions generated from 10,000 random permutations from data sets of 20 participants (see Methods for details). Original t test values (Attended vs. Unattended time at Unattended (left) and Attended (right) Locations) are indicated by the black vertical lines.

To complement the analysis of perceptual sensitivity, and to test whether the improvements came at the expense of longer response times, we also analyzed response times. Analogously to the analyses describe above, inverse efficiency values were submitted to a repeated-measures ANOVA with spatial and temporal expectations at the short interval as factors (Figure 2c). The interaction between spatial and temporal expectations was also significant, F(1, 19) = 7.83, p = 0.015; F-test permutation, p = 0.002. Similar to the effects in visual discrimination, temporal expectation only reduced response times when targets occurred at the expected spatial location (t-test permutation, p = 0.002). When targets were presented at the unexpected location, there was no effect of temporal expectation (t-test permutation p = 0.218). This analysis also revealed a main effect of spatial expectation, F(1, 19) = 51.73, p = 8.63 × 10⁻⁶; F-test permutation, p < 10⁻⁴, indicating that participants were faster when targets appeared at the expected location. No main effect of temporal expectation was found, F(1, 19) = 0.002, p = 0.964; F-test permutation, p = 0.962. 

Discussion

Spatial and temporal expectations are both important determinants of perception. However, so far, a unified theoretical framework explaining both types of effects has been lacking. In the current experiment we tested the recent proposal that temporal expectation interacts with predictions about spatial receptive-field properties to improve visual perception (Doherty et al., 2005; Nobre & Rohenkohl, 2014; Rohenkohl & Nobre, 2011). 

Confirming a critical prediction of this proposal, our results provide the first behavioral demonstration that temporal expectation combines synergistically with spatial expectation to enhance perceptual discriminability of visual events. On its own, temporal expectation was ineffective at improving perceptual functions in our task. When targets appeared at unexpected locations, there was no benefit of temporal expectation. However, when temporal expectation combined with spatial expectation, it significantly boosted perceptual benefits to sharpen perception. The pattern of results was consistent across the observers included in our study. Nonparametric, cluster-based analyses ensured that our pattern of results was not driven by unequal statistical power between conditions. 

The ability of temporal expectation to capitalize on receptive field-based biases is consistent with existing neurophysiological evidence. Doherty and colleagues (2005) showed that temporal expectation modulated the magnitude of visual P1 potentials only when temporal predictions were combined with spatial ones. Isolated temporal expectation had no effect on the first visual potential (see also Correa & Nobre, 2008). Neurophysiological studies in nonhuman primates have also noted time-modulation of effects of motor preparation (de Hemptinne, Nozaradan, Duvivier, Lefevre, & Missal, 2007; Lucchetti & Bon, 2001; Riehle, Grun, Diesmann, & Aertsen, 1997; Roux, Mackay, & Riehle, 2006; Schoffelen, Oostenveld, & Fries, 2005) and of spatial attention by temporal expectations (for reviews see Nobre, 2010; Nobre & Rohenkohl, 2014). Temporal expectation has been shown to modulate neuronal activity across several visual areas (Anderson & Sheinberg, 2008; Ghose & Bearl, 2010; Ghose & Maunsell, 2002), including V1 (Lima et al., 2011; Shuler & Bear, 2006). It is important to note that these studies have only manipulated temporal expectations about events occurring at predicted locations. Crossing temporal and spatial expectations in future intracranial recording studies will be particularly informative in revealing the degree of synergistic interactions between temporal and spatial expectations across different sensory cortices and subcortical areas. 

Synergistic interactions between temporal and spatial expectations are also consistent with perceptual benefits observed in behavioral experiments. When perceptually demanding tasks are used, temporal expectations are reported to enhance behavioral measures linked with perceptual stages of analysis. For example, Correa and colleagues (2005) found that symbolic temporal cues enhanced perceptual sensitivity (d′) to target letters in RSVP streams (see also Davranche et al., 2011). However, most previous behavioral experiments have also not manipulated temporal expectation independently of spatial expectation. Effects of temporal expectation on perceptual variables are often reported for foveal or spatially attended items (Lasley & Cohn, 1981; Jepma, Wagenmakers, & Nieuwenhuis, 2012; Marchant, Ruff, & Driver, 2012; Naccache, Blandin, & Dehaene, 2002; Westheimer & Ley, 1996). 

Having demonstrated the added benefits of combining temporal expectation with spatial expectation in vision, an important next step will be to characterize the psychophysical mechanisms by which combined spatial and temporal expectations bias perception. In two recent experiments, we have shown that a rhythmic temporal structure of external events increases the signal-to-noise efficiency for perceptual discrimination of targets (Cravo et al., 2013; Rohenkohl et al., 2012). However, in both studies targets were presented at fixed spatial locations, and the effects of temporal and spatial expectations could not be dissociated. 

Intriguingly, two recent studies (Vangkilde, Coull, & Bundesen, 2012; Vangkilde, Petersen, & Bundesen, 2013) suggest that, in addition to synergistic effects, temporal expectation may be capable of modulating visual processing under low spatial certainty conditions in some circumstances. They showed that temporal expectations improved perceptual processing in a divided spatial attention setting. In their tasks, relevant stimuli could be presented in one of two locations. Temporal expectation was modulated by an increase in hazard rates (Vangkilde et al., 2012, 2013) or by symbolic cues that indicated when targets were more likely to occur (Vangkilde et al., 2012; experiments 2 and 3). The studies did not test for possible interactions between spatial and temporal expectations. We would predict that effects of temporal expectation in their tasks would be significantly magnified by the addition of spatial certainty. Nevertheless, their results clearly suggest that spatial certainty is not always required for temporal expectation to influence perception. 

How can we reconcile our views with their findings? Spatiotemporal synergies may not be the only mechanism through which the temporal structure of events affects perception. For example, alertness, or temporally modulated alerting effects, may also contribute to performance. Alternatively, synergies may still be at play even when spatial attention is divided. Conditions in Vangkilde and colleagues' (2012, 2013) studies may have allowed for temporal modulation of two spatial templates. According to recent evidence, division of spatial attention may involve rapid and discrete shifts of attentional focus (Hogendoorn, Carlson, VanRullen, & Verstraten, 2010; Itthipuripat, Garcia, & Serences, 2013). Thus, the results found by Vangkilde and colleagues (2012, 2013) could be due to moments in which spatial and temporal orienting of attention coincide, producing an attenuated synergistic effect. If this is the case, then, as in the present study, the benefits of temporal expectations in divided attention tasks should be restricted to stimuli presented at the attended locations. 

An outstanding question is whether synergistic interactions between spatial and temporal expectations also occur in other sensory modalities. Though temporal expectations also benefit performance in auditory (Bausenhart, Rolke, & Ulrich, 2007; Costa-Faidella, Baldeweg, Grimm, & Escera, 2011; Henry & Obleser, 2012; Jaramillo & Zador, 2011; Jones, 1976, 2010; Jones, Moynihan, MacKenzie, & Puente, 2002; Lange, Rosler, & Roder, 2003; Poeppel, 2003; Schnuerch, Kreitz, & Lange, 2013; Shen & Alain, 2011, 2012; Zion Golumbic et al., 2013) and multisensory (Besle et al., 2011; Gomez-Ramirez et al., 2011; Lakatos et al., 2008; Lange & Roder, 2006, 2010) tasks, the patterns of perceptual modulation can differ substantially between modalities. For example, so far, interactive spatiotemporal effects have not been observed in audition (Lange, 2012; Rimmele, Jolsvai, & Sussman, 2011). Instead, spatial and temporal expectations are reported to modulate auditory processing independently. We, and others, have previously noted that differences in the patterns of modulation by temporal and spatial expectations across the senses are entirely reasonable (Lange, 2012; Nobre, 2010). Most likely, vision and audition have evolved for sensing different information in the external environment, and they display complementary levels of spatial and temporal acuity. While the spatial resolution of vision in humans exceeds auditory localization abilities, audition is characterized by high temporal sensitivity and acuity, with temporal parameters of stimuli being coded from the earliest, subcortical processing stages (King & Nelken, 2009; Poeppel, 2003; Theunissen, 2003; Theunissen, Sen, & Doupe, 2000; Viemeister & Plack, 1993). 

Another interesting question is whether temporal expectations can act by tuning the timing of biases based on nonspatial properties of receptive fields. Can temporal expectation also exploit other sources of anticipatory biases, such as a stimulus feature, to enhance perception? A pioneering study by Kingstone (1992) suggests that such interaction might indeed exist. Using cues to indicate the timing and form (letter V or A) of target events, Kingstone (experiment 4) found that temporal expectation modulated reaction times when it was coupled with expectation about target form. Conversely, there was no benefit of temporal cues when targets containing unexpected features were presented. These findings suggest that temporal expectation can indeed combine with other, nonspatial stimulus properties to improve behavior. However, to our knowledge, there is no evidence indicating that this interaction can occur at the perceptual level. Additionally, it would also be important to test the degree of interaction between all different types of stimulus attributes—temporal-, spatial-, and feature-based. Such three- or more-way interactions seem ecologically plausible and indeed likely. Temporal expectations, as well as other types, rarely, if ever, occur in isolation. Instead, expectations often come bundled in everyday situations. 

There is precedence for investigating how multiple types of attention bias interact in the literature. An early study by Hillyard and Münte (1984) suggested that other types of anticipatory biases, such as those based on stimulus feature (color), can be modulated by spatial expectations (see also Harter, Aine, & Schroeder, 1982). Their results showed that ERP differences associated with color selection (N150–350) were larger for targets appearing at attended versus unattended locations. It is unclear why this fundamental question has lost traction, but more studies of this kind will be essential when working toward a comprehensive model of attention. 

Finally, it is important to realize that temporal expectations can be generated in different ways. Temporal expectations can be cued by instructive or predictive stimuli (Cotti, Rohenkohl, Stokes, Nobre, & Coull, 2011; Coull, Davranche, Nazarian, & Vidal, 2013; Coull & Nobre, 1998; Davranche et al., 2011; Naccache et al., 2002; Zanto et al., 2011) as in the present experiment. They can also be induced by rhythmic stimulation (Cravo et al., 2013; de Graaf et al., 2013; Jones, 1976, 2010; Jones et al., 2002; Large & Jones, 1999; Marchant et al., 2012; Rohenkohl et al., 2012; Rohenkohl & Nobre, 2011; Snyder & Large, 2005; Zanto, Snyder, & Large, 2006), by manipulating hazard rates (Cravo, Rohenkohl, Wyart, & Nobre, 2011; Cui, Stetson, Montague, & Eagleman, 2009; Janssen & Shadlen, 2005; Vangkilde et al., 2012), or by the regularity, duration, or sequence of intervals between stimuli in a task (Jepma et al., 2012; Los & Agter, 2005; Los, Knol, & Boers, 2001; Vallesi, Shallice, & Walsh, 2007). It is not guaranteed that temporal expectations coming from these various sources are all supported by the same neurophysiological mechanisms and have the same consequences for behavior. Some evidence suggests that mechanisms supporting temporal expectations generated by rhythms versus symbolic cues differ (Breska & Deouell, 2014; Coull & Nobre, 2008; Jepma et al., 2012; Lange, 2012, 2013; Rohenkohl, Coull, & Nobre, 2011; Trivino, Arnedo, Lupianez, Chirivella, & Correa, 2011; Trivino, Correa, Arnedo, & Lupianez, 2010). Future studies are necessary to investigate systematically the ability of these different sources of temporal expectation to interact with other types of expectations. 

In conclusion, our results suggest that cued temporal expectations combine synergistically with predictive information about stimulus location to bias target processing in a top-down manner. These findings emphasize the importance of investigating interactions between different types of prediction in information processing. 

Supplementary Materials

pdf S1 

Acknowledgments

This work was funded by the Wellcome Trust WT 089903 (A. C. N.). In addition, the research was supported by the National Institute for Health Research (NIHR) Oxford Biomedical Research Centre based at Oxford University Hospitals Trust Oxford University. J. P. was supported by Science without Borders (SwB) scholarship and Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). 

Commercial relationships: none. 

Corresponding authors: Gustavo Rohenkohl; Anna Christina Nobre. 

Email: [e-mail]; [e-mail]. 

Address: Department of Experimental Psychology, University of Oxford, Oxford, UK.