Temporal crowding and its interplay with spatial crowding

Abstract

Spatial crowding refers to impaired target identification when the target is surrounded by other stimuli in space. Temporal crowding refers to impaired target identification when the target is surrounded by other stimuli in time. Previously, when spatial and temporal crowding were measured in the fovea they were interrelated with amblyopic observers but almost absent with normal observers (Bonneh, Sagi, & Polat, 2007). In the current study we examined whether reliable temporal crowding can be found for normal observers with peripheral presentation (9° of eccentricity), and whether similar relations between temporal and spatial crowding will emerge. To that end, we presented a sequence of three displays separated by a varying interstimulus interval (ISI). Each display included either one letter (Experiments 1a, 1b, 1c) or three letters separated by a varying interletter spacing (Experiments 2a, 2b). One of these displays included an oriented T. Observers indicated the T's orientation. As expected, we found spatial crowding: accuracy improved as the interletter spacing increased. Critically, we also found temporal crowding: in all experiments accuracy increased as the ISI increased, even when only stimulus-onset asynchronies (SOAs) larger than 150 ms were included, ensuring this effect does not reflect mere ordinary masking. Thus, with peripheral presentation, temporal crowding also emerged for normal observers. However, only a weak interaction between temporal and spatial crowding was found.

Introduction

When a task-relevant stimulus is surrounded by other items, our ability to identify this stimulus is lower than when it is presented in isolation. This phenomenon is often referred to as “crowding” (e.g., Bouma, 1970; Pelli, Palomares, & Majaj, 2004; Toet & Levi, 1992). Spatial crowding refers to cases in which the target is flanked by other stimuli presented simultaneously with the target, and it has been studied extensively (see Whitney & Levi, 2011, for a recent review). The extensive examination revealed several principles that underlie the phenomena of spatial crowding. These principles include, among others, the facts that: crowding is reduced as the distance between the target and flankers increases (e.g., Bouma, 1970; Pelli et al., 2004; Toet & Levi, 1992); the distance at which the flankers no longer affect target identification increases as target eccentricity increases (e.g., Bouma, 1970; Latham & Whitaker, 1996; Strasburger, 2005; Toet & Levi, 1992; Yeshurun & Rashal, 2010); spatial crowding is asymmetric in the sense that a more peripheral flanker has a larger effect on target identification than a more central flanker (e.g., Banks, Larson, & Prinzmetal, 1979; Bex, Dakin, & Simmers, 2003; Krumhansl, 1977; Petrov, Popple, & McKee, 2007); and grouping factors play a decisive role in crowding (e.g., Livne & Sagi, 2007; Manassi, Sayim, & Herzog, 2012; Saarela, Westheimer, & Herzog, 2010). It is also reasonable to expect temporal crowding, which is impaired identification of a target when it is surrounded in time by other stimuli (i.e., stimuli that appear before and after the target at the same spatial location). However, considerably less is known about the temporal aspects of crowding than the spatial aspects (Lev, Yehezkel, & Polat, 2014). 

Recently, Bonneh, Sagi, and Polat (2007) examined whether both temporal and spatial crowding can be demonstrated with amblyopic and normal vision observers, and whether they are interrelated. They employed one temporal crowding measure and two spatial crowding measures. The temporal crowding measure included the presentation of a rapid sequence of digits that was either fast in the crowded condition or slow in the uncrowded condition. The task was to identify one of the digits denoted by its smaller size. Note that in the crowded condition (fast presentation rate), the stimulus-onset asynchrony (SOA) between the onsets of each digit in the sequence was 200 ms, which is considerably longer than the 100–150 ms limit of classical masking (e.g., Breitmeyer, 1984; Breitmeyer & Ogmen, 2000; Enns, 2004; Enns & Di Lollo, 2000; Gorea, 1987; Michaels & Turvey, 1979). As for the spatial crowding measures, one measure involved orientation identification of an E-like stimulus presented simultaneously with other items in the crowded condition or by itself in the uncrowded condition. With the other spatial crowding measure, observers had to determine the alignment of two central Gabor patches, while two pairs of flanking patches were presented at different distances. Importantly, all stimuli were presented at the fovea. Bonneh et al. (2007) found evidence of temporal crowding (i.e., worse performance in the fast than slow condition) with one group of observers—strabismic amblyopes. Moreover, with these observers, the spatial and temporal measures of crowding were highly correlated. However, reliable effects of temporal crowding did not emerge with the other groups of observers: anisometropic amblyopes and normal vision observers. 

In the current study we examined whether significant effects of temporal crowding can also be found with normal vision observers. Because strong spatial crowding is typically found only when the stimuli are presented at the periphery of the visual field (see Levi, 2008; Whitney & Levi, 2011, for a review; but also see Lev et al., 2014, for a demonstration of foveal crowding), we wondered whether the lack of temporal crowding with the normal observers of the Bonneh et al. (2007) study was due to the foveal presentation. Hence, here we examined whether strong temporal crowding can be found for normal observers with peripheral presentation (Experiments 1a, 1b, 1c), and if so, whether strong relations between temporal and spatial crowding will emerge for our normal vision observers (Experiments 2a, 2b), as Bonneh et al. (2007) found for their strabismic amblyopes. 

Experiments 1a, 1b, and 1c

Experiments 1a, 1b, and 1c were designed to test whether significant temporal crowding will be found with normal vision observers when the various stimuli are presented at the periphery of the visual field. To that end, we presented at an eccentricity of 9°, a sequence of three rotated letters separated by a varying interstimulus interval (ISI). One of these letters was the target (the letter T), and the task was to indicate the target's orientation (Figure 1). The three experiments were identical except for the following: (a) To obtain a more comprehensive view of the effect of ISI, Experiments 1b and 1c included a wider range of ISIs (100–400 ms) than those employed in Experiment 1a (150–400 ms). (b) The letters employed in Experiments 1a and 1c were darker than those employed in Experiment 1b, to avoid ceiling effects. (c) In Experiments 1a and 1b, the exposure duration of a single letter was adjusted individually to avoid ceiling or floor effects. This resulted in somewhat different durations range between the experiments: the range of durations in Experiment 1a was 20–60 ms with a mode of 30 ms; the range of durations in Experiment 1b was 10–80 ms with a mode of 20 ms. In Experiment 1c, display duration was fixed at 30 ms for all participants. (d) Experiment 1c included “baseline” trials (36% of total trials) in which only the target was presented (i.e., without preceding or succeeding stimuli). These baseline trials were identical to the rest of the trials, with the only difference being the absence of preceding and succeeding stimuli. 

Figure 1

 

A schematic example of the sequence of events in a single trial of Experiments 1a, 1b, and 1c. In the baseline trials of Experiment 1c only the target was presented. Note that in Experiments 1a and 1b the duration of the letters display was adjusted individually for each observer, while in Experiment 1c display duration was 30 ms for all observers. In the example depicted here the display duration is 30 ms, the target appears in the first display, and the sequence of letters appears to the right of fixation.

Critically, in all these experiments, relatively long SOAs were employed. The shortest ISI we employed was 150 ms in Experiment 1a and 100 ms in Experiments 1b and 1c. Thus, given the exposure duration of a single letter, all the SOAs between the onset of each letter and the onset of the previous or following letter were larger than 100–150 ms (except for one or two of the shortest ISIs with some observers). Because 100–150 ms is cited by many studies as the limit of classical masking, particularly with displays that include a single item and masks that overlap the target (e.g., Breitmeyer, 1984; Breitmeyer & Ogmen, 2000, 2006; Enns, 2004; Enns & Di Lollo, 2000; Gorea, 1987; Michaels & Turvey, 1979), the long SOAs we employed ensured that if an effect of ISI would emerge it would not merely reflect the classical forward or backward masking observed with short ISIs. If temporal crowding can also occur with normal vision observers, performance should deteriorate the closer in time the letters are to each other (i.e., shorter ISIs should result in worse performance). 

Methods

Observers

Fifteen observers participated in Experiment 1a, 16 observers participated in Experiment 1b, and 16 observers participated in Experiment 1c. Ten of the observers who participated in Experiment 1c also participated in one of the other experiments (five in Experiment 1a, and five in Experiment 1b). All the participants were students from the University of Haifa, with normal or corrected-to-normal vision, and all of them were naive to the purpose of the study. This study adhered to the Declaration of Helsinki. 

Stimuli and apparatus

The stimuli were presented using MATLAB and the Psychophysics Toolbox extensions (Brainard, 1997) on a 19 in. monitor of an IBM compatible PC (resolution: 1024 × 768, 85 Hz). Eye movements were monitored with an EyeLink 1000 eye tracker (temporal resolution 1000 Hz; SR Research, Ottawa, ON, Canada). The stimuli were three gray (Experiments 1a, 1c: 20.5 cd/m²; Experiment 1b: 22.2 cd/m²) capital letters (1° × 1°) presented sequentially on a darker background (18 cd/m²) at an eccentricity of 9° to the right or left of a black fixation cross (0.3° × 0.3°). The target was the letter T, and the other two letters were chosen randomly from all the letters in the English language, except for I, J, L, Q, and W. Each letter was independently oriented upright, inverted, or tilted 90° to the left or to the right (Figure 1). 

Procedure

Each trial started with the fixation cross. After 1000 ms, the sequence of three letters was presented to the right or left of fixation, with equal probability. All the letters were presented for the same duration. In Experiments 1a and 1b this duration was adjusted individually for each participant to avoid ceiling or floor effects (Experiment 1a: durations range 20–60 ms, mode 30 ms; Experiment 1b: durations range 10–80 ms, mode 20 ms). In Experiment 1c, display duration was 30 ms for all participants. The letters were separated by an ISI chosen randomly on each trial from several possible values (Experiment 1a: 150, 175, 200, 225, 250, 300, 350, and 400 ms; Experiments 1b and 1c: 100, 125, 150, 175, 200, 225, 250, 300, 350, and 400 ms). During the ISI only the fixation cross was present on the screen. Note that because display duration in Experiment 1c was always 30 ms, these ISIs correspond to SOAs of 130, 155, 180, 205, 230, 255, 280, 330, 380, and 430 ms. The target was either the first or the second letter in the sequence, and the task was to indicate the orientation of the target. On 36% of the trials of Experiment 1c only the target was presented. The possible temporal positions of the target within the trial on these baseline trials were identical to those in the rest of the trials. That is, the target in the baseline trials could either appear in what would have been the first display in the sequence (i.e., 1000 ms after fixation onset) or what would have been the second display with correspondingly varying ISI (i.e., in these cases, target onset followed 1000 ms + 30 ms + varying ISI, which correspond to fixation time + first display time + varying ISI). An auditory feedback followed the observers' response. The values of the various variables: target temporal order within the sequence (first display vs. second display), target orientation (0°, 90°, 180°, or 270°), sequence location relative to fixation (left vs. right), and ISI, were chosen randomly, but occurred equally often throughout the experimental session. Each observer participated in 64 practice trials and 512 experimental trials in Experiment 1a, 80 practice trials and 640 experimental trials in Experiment 1b, and 62 practice trials and 992 experimental trials in Experiment 1c. 

Results and discussion

One observer in Experiment 1b and another from Experiment 1c were excluded from all analyses, due to excessive eye movements (i.e., with these observers fixation was broken on more than 20% of the trials). Additionally, one observer from Experiment 1a participated in two experimental sessions because the accuracy of the first session, in which display duration was shorter, was too low (average accuracy = 0.55); only the second session of this observer was included in the analysis. Thus, the final statistical analyses included 15 observers in each experiment, and for these observers, only trials on which fixation was not broken were taken into account (average percentage of trials excluded: Experiment 1a: 4.93%; Experiment 1b: 6.23%; Experiment 1c: 3.86%). 

A repeated-measures two-way ANOVA (ISI × target temporal order) was conducted on the accuracy data of each experiment (for Experiment 1c this analysis did not include the baseline trials). A significant main effect of ISI was found for all three experiments: Experiment 1a, F(7, 98) = 2.82, p < 0.02, η_p² = 0.17; Experiment 1b, F(9, 126) = 9.89, p < 0.0001, η_p² = 0.41; and Experiment 1c, F(9, 126) = 11.96, p < 0.0001, η_p² = 0.46. Figure 2 depicts performance as a function of ISI, and for Experiment 1c also as a function of SOA (which is simply the ISI + 30 ms). As can be seen in this figure, accuracy increased as the ISI between the letters increased up to about ISI of 300 ms. Critically, the effect of ISI was significant: Experiment 1b, F(7, 98) = 2.39, p < 0.03, η_p² = 0.15; and Experiment 1c, F(7, 98) = 7.33, p < 0.0001, η_p² = 0.34, even when the analysis of Experiments 1b and 1c did not include the two shortest ISIs (100, 125 ms), whose corresponding SOAs could have been shorter than 150 ms, and therefore may still reflect classical masking¹. 

Figure 2

 

Accuracy as a function of ISI in (a) Experiment 1a, (b) Experiment 1b, and (c) Experiment 1c. In Experiment 1c display duration was fixed (30 ms); thus, the SOAs that correspond to the ISIs are also presented. Error bars correspond to one standard error. BL = baseline trials in which only the target was presented.

The main effect of target temporal order was only significant in Experiment 1a, F(1, 14) = 5.05, p < 0.05, η_p² = 0.26, and the interaction between temporal order and ISI was significant in Experiment 1b, F(9, 126) = 3.61, p < 0.0006, η_p² = 0.21, and marginally significant in Experiment 1c, F(9, 126) = 1.74, p < 0.087, η_p² = 0.11.² As can be seen in Figure 3, with long ISIs performance was better when the target appeared in the first than the second display in the sequence, while with short ISIs, performance was better in the second display. This interaction implies that different processes may mediate the effect of ISI with short and long ISIs, but further research is required before any conclusive remark can be made regarding this pattern of results. 

Figure 3

 

Accuracy as a function of ISI and target temporal order (first display vs. second display) in (a) Experiment 1a, (b) Experiment 1b, and (c) Experiment 1c. Error bars correspond to one standard error.

To compare performance with preceding and succeeding stimuli to that without such stimuli we conducted a repeated-measures one-way ANOVA (ISI) on the accuracy data of Experiment 1c with the baseline trials included as one of the values of the ISI variable (i.e., this variable of ISI included 11 values: 10 ISIs + baseline). This analysis revealed a significant effect of ISI, F(10, 140) = 20.80, p < 0.0001, η_p² = 0.60; Figure 2c. Importantly, post hoc analysis with Bonferroni correction indicated that with all ISIs shorter than 300 ms (i.e., shorter than SOA of 330 ms) accuracy was significantly worse than in the baseline (p < 0.003, for all comparisons). This further demonstrates the long-lasting effects of ISI we term temporal crowding. With ISIs equal or longer than 300 ms accuracy did not significantly differ from baseline. Finally, we also analyzed accuracy in the baseline trials as a function of target temporal position within the trial. We created a new independent variable we called “temporal position” and it included 11 values: 1 value corresponded to baseline trials in which the target appeared at what would have been the first display, and 10 values corresponded to baseline trials in which the target appeared in what would have been the second display, with 1 value for each possible ISI. We then conducted a repeated-measures one-way ANOVA (temporal position) on the accuracy of the baseline trials. This analysis indicated that accuracy did not vary as a function of temporal position, F(10, 140) = 0.62, p = 0.79. Thus, in the absence of preceding and succeeding stimuli, the temporal position of the target within the trial sequence does not matter. 

To sum, when the sequence of stimuli is presented to the periphery of the visual field temporal crowding emerges also for observers with normal vision: performance was worst when other items appeared before or after the target within a short time interval and it improved as the stimuli were separated by a longer time interval. Optimal performance was observed when the stimuli were separated by 300 ms. 

Experiments 2a and 2b

Experiments 1a, 1b, and 1c demonstrated that a significant temporal crowding can be found with normal observers when the sequence of stimuli is presented to the periphery of the visual field. Experiments 2a and 2b were designed to examine the relations between temporal crowding and spatial crowding. As mentioned above, Bonneh et al. (2007) found strong relations between temporal and spatial crowding for strabismic amblyopes with foveal presentation. Here we tested whether similar relations will be found for normal vision observers with peripheral presentation. Thus, the experimental procedure in these experiments was similar to that in Experiments 1a, 1b, and 1c: peripheral presentation of a sequence of three displays separated by a varying ISI. However, unlike the previous experiments, in which only a single letter appeared in each display, in Experiments 2a and 2b each display included three letters—a central letter flanked by two other letters. The three letters were separated by a varying interletter spacing (Figure 4). The ISI between the displays and the spacing between the letters varied independently. Like before, the target was the letter T, and it could be one of the central letters of one of the three displays in the sequence. The task was to indicate the T's orientation. The two experiments were identical except for the following: (a) To obtain a more comprehensive view of possible interactions between the effects of ISI and interletter spacing while keeping an experimental session within reasonable length, Experiment 2a included six possible values of interletter spacing and three ISIs, while Experiment 2b included three possible values of interletter spacing and six ISIs; and (b) As in the previous experiments, the exposure duration of each of the letters display was adjusted individually to avoid ceiling or floor effects, which resulted in somewhat different durations range between the experiments (Experiment 2a: durations range 20–60 ms, mode 30 ms; Experiment 2b: durations range 20–80 ms, mode 20 ms). 

Figure 4

 

A schematic example of the sequence of events in a single trial of Experiments 2a and 2b. Note that the duration of the letters display was adjusted individually for each observer. In the example depicted here, the display duration is 30 ms, the target appears in the second display, and the sequence of letters appears to the left of fixation.

Given the findings of Experiments 1a, 1b, and 1c, we expected to find here a similar indication of temporal crowding: performance should deteriorate the closer in time the letters displays are to each other (i.e., shorter ISIs should result in worse performance). Additionally, we expected to find the pattern of results that is typical to spatial crowding: performance should deteriorate the closer in space the flankers are to the target (i.e., smaller interletter spacing should result in worse performance). Finally, if temporal and spatial crowding are also strongly related with normal vision observers and peripheral presentation, a significant interaction between the factors of ISI and interletter spacing should emerge. 

Methods

Observers

Fifteen observers participated in Experiment 2a and 18 observers participated in Experiment 2b. All the observers of Experiment 2a participated only in this experiment. Five observers from Experiment 2b also participated in one or more of the other experiments (two in Experiments 1a and 1c, and three in Experiments 1b and 1c). All the observers were students from the University of Haifa, with normal or corrected-to-normal vision, and all of them were naive to the purpose of the study. 

Stimuli, apparatus, and procedure

The stimuli, apparatus, and procedure were identical to those of Experiment 1b except each letter display included three letters. The central letter was presented at fixation level, at an eccentricity of 9° to the right or left. The other letters were presented above and below the target with the same interletter spacing (Figure 4). The interletter spacing was chosen randomly on each trial from one of the following six possible values in Experiment 2a: 2°, 3°, 4°, 5°, 6°, and 7°; and one of the following three possible values in Experiment 2b: 2°, 4°, and 6°. On 14.3% of the trials of each experiment, only a single letter was presented in each display (no-spatial-flankers condition). The ISI between the letters displays was also chosen randomly on each trial. Experiment 2a included three possible ISIs: 150, 250, and 450 ms; and Experiment 2b included six possible ISIs: 125, 150, 200, 250, 300, and 350 ms. The target was the central letter in one of the three displays in the sequence. Like before, display duration was adjusted individually for each participant (Experiment 2a: duration range 20–60 ms, mode 30 ms; Experiment 2b: duration range 20–80 ms, mode 20 ms). The values of the various variables: target temporal order within the sequence (first display, second display, or third display), target orientation (0°, 90°, 180°, or 270°), sequence location relative to fixation (left vs. right), ISI, and interletter spacing were chosen randomly, but occurred equally often throughout the experimental session. Each observer participated in 48 practice trials and 1,008 experimental trials. 

Results and discussion

Three observers from Experiment 2b were excluded from the analysis due to excessive eye movements (i.e., they broke fixation on more than 20% of the trials). With the other observers, 15 in each experiment, only trials in which fixation was not broken were included in the statistical analyses (average percentage of trials excluded: Experiment 2a: 7.78%; Experiment 2b: 7.23%). 

A repeated-measures three-way ANOVA (ISI × interletter spacing × target temporal order) was conducted on the accuracy data of each experiment. A significant main effect of ISI was found for both experiments: Experiment 2a, F(2, 28) = 10.08, p < 0.0006, η_p² = 0.42; Experiment 2b, F(5, 70) = 16.46, p < 0.0001, η_p² = 0.54. As in Experiments 1a, 1b, and 1c, accuracy increased as the ISI between the displays increased (Figure 5), and the effect of ISI was significant, F(4, 56) = 7.4, p < 0.0001, η_p² = 0.35, even when the analysis of Experiment 2b did not include the shortest ISI (125 ms). Thus, here too, temporal crowding emerged for normal vision observers when the stimuli were presented at the periphery. 

Figure 5

 

Accuracy as a function of ISI in (a) Experiment 2a and (b) Experiment 2b. Error bars correspond to one standard error.

The main effect of interletter spacing was also significant in both experiments: Experiment 2a, F(6, 84) = 71.71, p < 0.0001, η_p² = 0.84; Experiment 2b, F(3, 42) = 107.80, p < 0.0001, η_p² = 0.89; accuracy increased as the spacing between the three letters of each display increased (Figure 6). Thus, as expected, spatial crowding also emerged with peripheral presentation. Critically, a significant interaction between ISI and interletter spacing was found only with Experiment 2b, F(15, 210) = 2, p < 0.02, η_p² = 0.12, and, as can be seen in Figure 7, this interaction is due only to the shortest ISI (125 ms); the effect of interletter spacing was more pronounced with this ISI than with all the other ISIs. Indeed, when this ISI is taken out of the analysis, the interaction is no longer significant, p = 0.79. Hence, when we only consider long ISIs that do not involve ordinary masking there is no interaction between the two factors. This suggests that with normal vision observers spatial and temporal crowding do not interact regardless of whether the presentation is foveal as in Bonneh et al. (2007) or peripheral as in our experiments. 

Figure 6

 

Accuracy as a function of interletter spacing in (a) Experiment 2a and (b) Experiment 2b. nf = no-spatial-flankers condition. Error bars correspond to one standard error.

Figure 7

 

Accuracy as a function of ISI and interletter spacing in (a) Experiment 2a and (b) Experiment 2b. nf = no-spatial-flankers condition. To avoid clutter, only a single error bar is presented corresponding to the averaged standard error.

The main effect of target temporal order was only significant in Experiment 2a, F(2, 28) = 7.19, p < 0.004, η_p² = 0.34; Bonferroni post hoc analysis indicated that accuracy was significantly lower, p < 0.02, when the target appeared in the first display (0.78, SD = 0.42) than in either of the other two displays (second, 0.81, SD = 0.39; third, 0.82, SD = 0.38). There was no significant difference between the second and third displays. The interaction between target temporal order and ISI was significant in both experiments: Experiment 2a, F(4, 56) = 7.44, p < 0.0001, η_p² = 0.35; Experiment 2b, F(10, 140) = 6.07, p < 0.0001, η_p² = 0.30. The pattern of this interaction was very similar in both experiments (Figure 8): with the two initial temporal positions (i.e., when the target appeared in the first or second display), accuracy improved as the ISI increased. However, there was no effect of ISI when the target appeared in the last temporal position (i.e., the third display). 

Figure 8

 

Accuracy as a function of ISI and target temporal order (first display, second display, or third display) in (a) Experiment 2a and (b) Experiment 2b. Error bars correspond to one standard error.

The interaction between target temporal order and interletter spacing was also significant with both experiments: Experiment 2a, F(12, 168) = 2.33, p < 0.009, η_p² = 0.14; Experiment 2b, F(6, 84) = 5.69, p < 0.0001, η_p² = 0.29. Again, the pattern of the interaction is similar in both experiments (Figure 9): with all temporal positions (i.e., regardless of the display in which the target appeared) accuracy increased as the interletter spacing increased, however, this effect of interletter spacing was more pronounced with the initial position than the last position. 

Figure 9

 

Accuracy as a function of interletter spacing and target temporal order (first display, second display, or third display) in (a) Experiment 2a and (b) Experiment 2b. nf = no-spatial-flankers condition. Error bars correspond to one standard error.

Finally, the three-way interaction (ISI × interletter spacing × target temporal order) was significant in Experiment 2a, F(24, 336) = 1.71, p < 0.03, η_p² = 0.11, and just above the marginal-significance criterion in Experiment 2b, F(30, 420) = 1.36, p = 0.1, η_p² = 0.09. The interaction for both experiments is presented in two different ways, with ISI as the x-axis in Figure 10 and interletter spacing as the x-axis in Figure 11. To investigate the source of this interaction we performed three separate two-way ANOVA (ISI × interletter spacing) for each of the possible target temporal positions. As expected, the outcomes of these analyses follow the pattern of results reflected in the main effects and two-way interactions of the main analysis described above. With both experiments the effect of ISI was significant when the target appeared in the first or second display but not when it appeared in the third display: Experiment 2a, first, F(2, 28) = 20.41, p < 0.0001, η_p² = 0.59; second, F(2, 28) = 6.06, p < 0.007, η_p² = 0.30; third, F(2, 28) = 1.08, p = 0.35; Experiment 2b, first, F(5, 70) = 15.13, p < 0.0001, η_p² = 0.52; second, F(5, 70) = 12.95, p < 0.0001, η_p² = 0.48; third, F(5, 70) = 0.81, p = 0.55. Also, with both experiments, the effect of interletter spacing was significant with all possible temporal positions: Experiment 2a, first, F(6, 84) = 43.94, p < 0.0001, η_p² = 0.76; second, F(6, 84) = 50.97, p < 0.0001, η_p² = 0.78; third, F(6, 84) = 21.27, p < 0.0001, η_p² = 0.60; Experiment 2b, first, F(3, 42) = 96.24, p < 0.0001, η_p² = 0.87; second, F(3, 42) = 55.01, p < 0.0001, η_p² = 0.80; third, F(3, 42) = 71.37, p < 0.0001, η_p² = 0.84. Interestingly, these analyses also revealed the fact that in both experiments a two-way interaction (ISI × interletter spacing) emerged only when the target appeared in the second display, and this interaction was significant in Experiment 2b, F(15, 210) = 2.01, p < 0.02, η_p² = 0.13, and marginally significant in Experiment 2a, F(12, 168) = 1.72, p = 0.066, η_p² = 0.11. As can be seen in Figures 10 and 11, when the target appeared in the first display the accuracy increment with increasing ISI is present with almost all values of interletter spacing, and when the target appeared in the third display the accuracy increment with increasing ISI is absent with all interletter spacing. However, when the target appeared in the second display there is more variability. In general, the accuracy increment with increasing ISIs seems to be more pronounced with the smaller rather than the larger spacing and vice versa—the accuracy increment with larger interletter spacing is more pronounced with shorter rather than longer ISIs. Still, these differences are rather small, and the smaller ISI effect with the larger spacing and smaller spacing effect with longer ISIs may be due to ceiling effects. Thus, an interaction between spatial and temporal crowding can be found with normal vision observers, but only for targets presented in the middle of the temporal sequence, and it appears to be a rather weak interaction. 

Figure 10

 

Accuracy as a function of ISI, interletter spacing, and target temporal order in Experiment 2a when the target is in: (a) first display; (b) second display; and (c) third display; and in Experiment 2b when the target is in: (d) first display; (e) second display; and (f) third display. nf = no-spatial-flankers condition. To avoid clutter, only a single error bar is presented corresponding to the averaged standard error.

Figure 11

 

Accuracy as a function of interletter spacing, ISI and target temporal order in Experiment 2a when the target is in: (a) first display, (b) second display, and (c) third display; and in Experiment 2b when the target is in: (d) first display, (e) second display, and (f) third display. nf = no-spatial-flankers. To avoid clutter, only a single error bar is presented corresponding to the averaged standard error (same data as in Figure 10).

To ensure that the lack of ISI × interletter spacing interaction is not due to the variations in display duration we performed two additional analyses. First, because these experiments differed only in their ISI and spacing values we could combine their data and examine the effects of ISI and spacing only for observers who run with the same duration. This analysis only included ISI and spacing values that were common to both experiments. Additionally, we combined the ISIs of 450 ms and 350 ms (from Experiments 2a and 2b, respectively) into a single value because with both ISIs accuracy was already at asymptotic level. The validity of this action is confirmed by the fact that the effect of ISI did not vary significantly as a function of experiment (p = 0.35). The display duration of 20 ms was the most frequent duration of the combined data (11 observers). We, therefore, performed a repeated-measures two-way ANOVA (ISI × interletter spacing) on the data of these 11 observers. This analysis revealed a very similar pattern of results to those found with the individual analyses of Experiments 2a and 2b, described above. That is, the main effects of ISI and spacing were significant, F(2, 20) = 5.53, p < 0.02 and F(3, 30) = 54.26, p < 0.0001, respectively, but their interaction was not (p = 0.46). The same pattern of results emerged when we performed an identical analysis on the second most prevalent display duration: 30 ms with 10 observers: ISI, F(2, 18) = 5.32, p < 0.02; spacing, F(3, 27) = 54.62, p < 0.0001; ISI × spacing, p = 0.56. Other display durations included too few people. Thus, the same pattern of results emerged when the duration varied and when it did not. The second test of possible duration effects took advantage of the fact that, in the combined data, there was only one display duration that was unique to a single observer—80 ms. We could, therefore, simply exclude this observer, and then perform on the data of all remaining 29 observers a three-way mixed-design ANOVA with ISI and spacing as within-subjects variables, and duration as a between-subjects variable. Regarding ISI and spacing, this analysis, again, revealed a very similar pattern of results to that described above: The main effects of ISI and spacing were significant, F(2, 48) = 8.08, p < 0.001 and F(3, 72) = 143.29, p < 0.0001, respectively, but their interaction was not (p = 0.39). Critically, the main effect of duration and its interactions with other factors were not significant (p > 0.1 for all). Hence, display duration was not an influential factor. 

General discussion

In this study we examined whether temporal crowding can be found for observers with normal vision when the stimuli are presented to the periphery of the visual field, and if so, whether strong relations will be found between temporal and spatial crowding. The study included five experiments. Three experiments (Experiments 1a, 1b, and 1c) employed a sequence of three displays separated by a varying ISI, in which each display included only a single letter. One of the letters was the target. The other two experiments (Experiments 2a and 2b) employed a similar sequence of three displays; however, in these experiments each display included three letters with varying interletter spacing. A significant effect of ISI was found in all five experiments. Accuracy increased as the ISI increased, reaching optimal performance around ISI of 300 ms. 

Importantly, the effect of ISI was significant even when the analysis included only ISIs that are longer than the reported limits of classical backward and forward masking (e.g., Breitmeyer, 1984; Breitmeyer & Ogmen, 2000, 2006; Enns, 2004; Enns & Di Lollo, 2000; Gorea, 1987; Michaels & Turvey, 1979; Scheerer, 1973). The masking literature distinguishes between different types of masks (e.g., Breitmeyer, 1984; Breitmeyer & Ogmen, 2000; Di Lollo, Enns, & Rensink, 2000; Enns, 2004). Masking by light refers to masks composed of a uniform flash of light that spatially overlaps the target. Masking by pattern or structure also refers to masks that overlap the target, but in this case the mask and target share some features; this type of masking is the one relevant for our study. Masking by noise is another type of masks that overlap the target but these masks are composed of random-dot noise. Finally, metacontrast and paracontrast masking refer to masks that do not spatially overlap the target (for further details see Breitmeyer, 1984; Breitmeyer & Ogmen, 2000). Enns (2004) compared the characteristics of different types of masks and found that for set-size one (i.e., when only a single target was present, as was the case in the current Experiments 1a, 1b, and 1c) a very typical pattern of results emerged for all masking types: masking was over by the time 150 ms has elapsed between the presentation of the target and that of the mask. The fact that, in our experiments, an effect of ISI emerged with ISIs larger than 150 ms suggests that this effect persists beyond the limit of classical masking. We term this long-lasting effect “temporal crowding”; however, at this point we cannot tell whether this effect reflects processes that are different from those mediating classical masking or whether the same processes mediate both classical masking and temporal crowding. 

In the spatial domain, the question of whether crowding and masking are the same thing or different phenomena was extensively debated. Some studies suggest that the two phenomena are related (e.g., Lev et al., 2014; Polat, Sterkin, & Yehezkel, 2007), but others presented evidence suggesting that these are two different phenomena (e.g., Chakravarthi & Cavanagh, 2009; Levi, Hariharan, & Klein, 2002; Parkes, Lund, Angelucci, Solomon, & Morgan, 2001; Pelli et al., 2004; Pelli & Tillman, 2008; Petrov et al., 2007). For instance, Pelli and colleagues (2004) suggested that with masking the target disappears, but with crowding the target remains visible but becomes obscure. Additionally, Pelli et al. (2004) and Parkes et al. (2001) suggested that unlike masking, the spatial extant of crowding depends on the eccentricity. Petrov and colleagues (2007) demonstrated that spatial crowding, unlike surround suppression, is asymmetric with the more peripheral flanker having a larger effect than the more central flanker. Finally, Chung, Levi, and Legge (2001) found that on some aspects spatial crowding and pattern masking are only quantitatively different (e.g., crowding demonstrates broader spatial frequency tuning). Still, on other aspects, like the fact that the spatial extent of crowding is independent of letter frequency, the two phenomena are qualitatively different. Chung et al. (2001) concluded that spatial crowding and pattern masking likely share the early stages of processing but may differ on later stages. Similar questions can be asked with regard to the temporal domain, but considerable additional research is required before any conclusion can be reached. The only evidence we have thus far in support of the possibility that the effect observed with short ISIs is qualitatively different than the effect observed with long ISIs is the two-way interaction (ISI × target temporal order) that was found in Experiments 1b and 1c. Specifically, an opposite effect of temporal order was found with the shorter ISIs in comparison to the longer ISIs. With the former, performance was better when the target appeared in the second display but with the latter, performance was better when the target appeared in the first display. However, this interaction was significant in Experiment 1b but only marginally significant in Experiment 1c, and it is not obvious what processes underlie these different patterns of temporal order effect. 

Experiment 1c, with its baseline (uncrowded) trials, allows us to rule out the possibility that the long-lasting effects of ISI found here are merely due to “absolute” temporal uncertainty that regards the temporal position of the target within a trial (i.e., uncertainty about target onset relative to fixation onset). This is because such uncertainty was present for both uncrowded and crowded trials; hence, such uncertainty cannot explain the significant differences found in this experiment between crowded and uncrowded trials. Following the same logic, Experiment 1c also allows us to rule out the possibility that these long-lasting ISI effects are merely due to spatial uncertainty (i.e., uncertainty about the location in space of the stimuli, right vs. left), because spatial uncertainty was present for both uncrowded and crowded trials, and therefore it cannot explain the significant differences found between these two types of trials. However, the current findings do not allow us to determine the role played by “relative” temporal uncertainty that regards the target position within the sequence of items. That is, we cannot tell whether these long-lasting effects will emerge when target position is fixed within the sequence of three displays. Additionally, the current findings do not mean that spatial uncertainty or absolute temporal uncertainty play no role in temporal crowding, only that these are not the main factors mediating the effect. In fact, we have previously demonstrated that reducing spatial uncertainty via a peripheral cue that attracts transient attention to the target location reduced spatial crowding. Still, precueing the target location did not completely overcome spatial crowding (Rashal & Yeshurun, 2014; Yeshurun & Rashal, 2010). The current findings cannot tell us whether or not a reduction in spatial uncertainty can also reduce temporal crowding. Hence, a detailed account of the role played by temporal and spatial uncertainties in temporal crowding awaits further research. 

The fact that in our experiments an effect of ISI emerged with ISIs longer than 150 ms is consistent with Bonneh and colleagues' (2007) finding that strabismic amblyopes performed significantly worse with a presentation rate of 200 ms than 400 ms. This suggests that when the stimuli are presented to the periphery of the visual field observers with normal vision demonstrate similar temporal crowding to that found for strabismic amblyopes with foveal presentation. However, unlike the study of Bonneh et al. (2007), we did not find strong relations between temporal and spatial crowding. Bonneh et al. (2007) found that with strabismic amblyopes the measures of temporal and spatial crowding were highly correlated. To examine whether these two types of crowding interact with normal vision observers with peripheral presentation, we simultaneously manipulated the ISI between the displays and the interletter spacing within each display (Experiments 2a and 2b). An interaction between these two factors emerged only when the target appeared in the second display and only with the shortest ISI of Experiment 2b (125 ms), which may still reflect ordinary masking. Thus, based on the current findings we can only conclude that with normal vision observers, only a weak interaction exists between these two factors, showing a weaker effect of ISI with larger interletter spacing and vice versa. This may imply that different processes underlie the foveal temporal crowding found with strabismic amblyopes and the peripheral temporal crowding found here with normal observers. 

Still, the lack of strong interaction between spatial and temporal crowding does not mean that temporal factors cannot affect spatial crowding. In fact, effects of temporal factors on spatial crowding were already demonstrated. For instance, Vickery, Shim, Chakravarthi, Jiang, & Luedeman (2009) found that spatial crowding extends over a larger area when a mask follows the target after a very short SOA (26.7 or 53.3 ms). They termed this phenomenon “supercrowding” and suggested that it is due to the additional noise generated by the mask. Chakravarthi and Cavanagh (2007) examined the polarity advantage effect—the fact that spatial crowding decreases when the target and flankers have opposite contrast polarity. They found that this advantage is eliminated when the stimuli reverse their polarity at a relatively fast rate (7.5 Hz or higher). Because this rate is similar to the limit at which attention can individuate the alternating phases of flickering components, they suggested that their finding supports the hypothesis that spatial crowding reflects the resolution limitation of attention. Several studies have also demonstrated such effects by varying the relative onset of target and flankers (e.g., Harrison & Bex, 2014; Huckauf & Heller, 2004; Ng & Westheimer, 2002). Huckauf and Heller (2004), for example, varied the SOA between the onset of the target and its adjacent flankers, with both negative and positive SOAs. They found that in general spatial crowding decreased as the SOA between the target and flankers increased. However, the pattern of this SOA effect varied as a function of the target-flankers spacing and target eccentricity. Specifically, they found that with small spacing and large eccentricities, crowding decreased monotonically as the SOA increased. However, with larger spacing and small eccentricities, a nonmonotonic function emerged—crowding was most detrimental when the flankers followed the target by 50 ms (i.e., a positive SOA of 50 ms), rather than when the target and flankers appeared simultaneously. Huckauf and Heller (2004) interpreted these findings to indicate that with small spacing and large eccentricities spatial crowding is mediated by faulty integration of the flankers and target, while with larger spacing and smaller eccentricities, crowding effects reflect the involvement of higher feedback processes that interfere with target processing. Nonmonotonic effects were also observed by Ng and Westheimer (2002) and Harrison and Bex (2014). Both studies found the largest crowding effects when the flankers followed the target by about 50 ms. Additionally, Harrison and Bex (2014) also included a condition in which the flankers changed their identity before target onset or after its offset. They found that the crowding elevation found with trailing flankers was further intensified when the flankers changed their identity. In contrast, the crowding alleviation found for preceding flankers was eliminated when the flankers changed their identity. Similar to Huckauf and Heller (2004), Harrison and Bex (2014) also suggested that their findings reflect the involvement of higher level processing. These various findings clearly demonstrate that spatial crowding is effected by temporal factors. However, they do not provide information regarding temporal crowding because in all these examples the stimuli that preceded or succeeded the target were always presented in adjacent locations, never at the target location, as was done in the current study. 

Another example of temporal factors affecting spatial crowding was provided recently by Lev et al. (2014), who measured the extent of spatial crowding while varying the duration of the display. They found that when the target was flanked by other stimuli, performance was considerably worse with shorter presentation durations. In contrast, when the target was presented without flankers, only a modest effect of stimulus duration was found. These findings demonstrate the relevance of the temporal characteristics of the display to performance under spatial crowding conditions. Interestingly, Lev and colleagues' (2014) study also provides some evidence of an interaction between spatial and temporal crowding. In one of their experiments, a backward mask followed the target with a varying ISI. They found that when the target was presented without flankers there was no effect of ISI with SOAs longer than 120 ms (i.e., an ISI of 90 ms for the 30 ms duration condition, and an ISI of 60 ms for the 60 ms duration condition). However, when the target was presented together with flankers, the effect of ISI lasted longer; with the 60 ms duration, performance did not reach baseline level even with the longest ISI employed (120 ms). Although the target in their study was always presented at the center of the display, unlike our peripheral presentation, this finding suggests that the two types of crowding do interact even with normal vision observers but this depends on the specific experimental conditions involved. Hence, additional experimental work is required to establish the conditions that reveal this interaction. 

Moreover, if the temporal crowding demonstrated in this study is viewed as a unique case of masking, then it is important to note that there are also several studies (see Herzog, 2007, for a review) demonstrating effects of the spatial characteristics of the display on masking, particularly backward masking. For instance, Herzog et al. (2003) have shown that various types of masks exert weaker masking effects when they extend over larger spatial areas. Additionally, masks composed of 25 random lines inflict stronger masking effects than masks composed of 25 regularly spaced lines. These findings underscore the relevance of the spatial domain to our understanding of backward masking. Such findings further suggest that the lack of strong interaction between temporal and spatial crowding observed in our current study may not reflect a general absence of such interaction, but rather the fact that such interaction may only emerge under specific conditions that are yet to be revealed. 

It is also interesting to note the finding that, like spatial crowding, temporal crowding is asymmetric. That is, with spatial crowding it is already established that a flanker that is more peripheral than the target leads to greater identification impairment than a flanker that is more central than the target (e.g., Bex et al., 2003; Krumhansl, 1977; Petrov et al., 2007). With temporal crowding we found that a stimulus that appears after the target has a stronger effect on target identification than a stimulus that appears before the target. Specifically, in both Experiments 2a and 2b, temporal crowding was found for targets appearing in the first or second displays. But when the target appeared in the third (last) display performance was not affected by the ISI. This finding suggests that the presence of a trailing stimulus (a stimulus that follows the target) might be a critical condition for the emergence of long-lasting ISI effects. Additionally, even though targets that appeared in the second display were surrounded by both preceding and succeeding stimuli, their identification was often better than that of targets appearing in the first display, which were not preceded by any stimulus. However, as already discussed, when the displays only included a single letter (Experiments 1a, 1b, and 1c), opposing effects were found with short and long ISIs. Thus, these various effects of temporal order also require further investigation. 

Finally, one may wonder what are the processes that underlie the observed temporal crowding. At this stage we can only speculate. Several studies suggested the distinction between integration masking and interruption masking (e.g., Di Lollo, Enns, & Rensink, 2000; Enns, 2004; Kahneman, 1968; Scheerer, 1973; Turvey, 1973). Integration masking refers to cases in which the target and the mask are combined into a single unit due to temporal resolution limitations. Integration masking occurs mainly with short target-mask SOAs (up to about 100 ms), and it likely reflects early levels of visual processing. Given these short SOAs, integration processes do not seem to be the ones mediating temporal crowding. Interruption masking is believed to occur at later stages. The onset of the mask disrupts the processing of the target before the target was fully processed. In other words, the mechanisms that process the target do not combine the signal of the target and mask as is the case with integration masking, but rather they start to process the mask instead of the target. Enns (2004) suggests that this later masking process is object substitution. According to the object substitution theory, reentrant processes mediate neural communication between higher brain centers that generate hypotheses regarding the identity of the target and lower brain regions that analyze the sensory input (e.g., Di Lollo et al., 2000; Enns & Di Lollo, 2000). If there is a match between the hypothesis generated by the higher-level mechanisms and the activity at the lower-level mechanisms, a stable representation of the target is established. However, if the mask replaces the target before a stable representation of the target was established, there will not be a match between the higher and lower mechanisms, and the iterative processing will start again, generating hypothesis for the new sensory input—the mask. The possibility that the temporal crowding effects found in the current experiments are due to such iterative processes that involve higher-level mechanisms seems more likely, given that the effects of ISI found here lingered for relatively long durations. Masking by object substitution is also consistent with the finding that, when the target appeared in the last display (i.e., it was not replaced by a mask), there was no effect of ISI. However, the theory predicts long-ranging effects of masking only when attention cannot be easily focused on the target; for example when the set-size is larger than one (i.e., when there is more than one item in each display). Yet, we found lingering effects of ISI also in Experiments 1a, 1b, and 1c, in which only a single letter was present in each display. Similarly, since the letters were always presented to the same location, the onset of the letter in the first display attracts spatial attention to its location, so by the time the second letter appears, attention is already focused on its location. Thus, the processing of the letter in the second display always involved spatial attention and therefore, according to the theory, long-lasting masking effects should not have emerged. Yet we found significant effects of ISI even for targets appearing in the second display. The object substitution theory in its current form, therefore, may not be the perfect explanation of our findings, but the basic idea that temporal crowding reflects substitution of target processing, at some higher level, by the item that follows it is nevertheless an attractive candidate. 

To summarize, this study demonstrates, for the first time, temporal crowding for normal vision observers with a peripheral presentation. Specifically, we have found that target identification deteriorates when other stimuli are presented with close temporal proximity to the target. Critically, the effects of temporal crowding lasts for longer ISIs than the classical forward and backward masking effects, and may therefore reflect different, possibly later, processes. Temporal crowding was found regardless of whether the target was presented alone or with additional flanking stimuli, and only a weak interaction emerged between spatial and temporal crowding. 

Acknowledgments

Commercial relationships: none. 

Corresponding author: Yaffa Yeshurun. 

Email: [e-mail]. 

Address: Department of Psychology and Institute of Information Processing and Decision Making, University of Haifa, Haifa, Israel. 

Footnotes

¹  Because SOA = ISI + display duration, and because 14 observers in Experiment 1b had display duration that is shorter than 50 ms, for these observers the SOA that corresponds to the ISI of 100 ms is shorter than 150 ms. Similarly, since 11 observers had display duration that is shorter than 25 ms, for these observers the SOA that corresponds to the ISI of 125 ms is also shorter than 150 ms. In Experiment 1c the display duration was always 30 ms, hence, the two shortest ISIs correspond to SOAs of 130 and 155 ms for all observers. We, therefore, reanalyzed the ISI effect for Experiments 1b and 1c while excluding these two ISIs (100, 125 ms). With Experiment 1a, this issue does not arise because the shortest ISI is 150 ms; hence, even with the shortest display duration employed in this experiment the corresponding SOA is larger than 150 ms.

Footnotes

²  Because display duration was fixed in Experiment 1c, we could not individually control performance level. This resulted in one participant demonstrating an overall accuracy of 97%, which is obviously at ceiling. If this participant is taken out of the analysis, this interaction becomes significant, F(9, 117) = 2.05, p < 0.04, η_p² = 0.14.