How the global layout of the mask influences masking strength

Abstract

Abstract

Abstract:

Abstract  In visual backward masking, the perception of a target is influenced by a trailing mask. Masking is usually explained by local interactions between the target and the mask representations. However, recently it has been shown that the global spatial layout of the mask rather than its local structure determines masking strength (Hermens & Herzog, 2007). Here, we varied the mask layout by spatial, luminance, and temporal cues. We presented a vernier target followed by a mask with 25 elements. Performance deteriorated when the length of the two mask elements neighboring the target vernier was doubled. However, when the length of every second mask element was doubled, performance improved. When the luminance of the neighboring elements was doubled, performance also deteriorated but no improvement in performance was observed when every second element had a double luminance. For temporal manipulations, a complex nonmonotonic masking function was observed. Hence, changes in the mask layout by spatial, luminance, and temporal cues lead to highly different results.

Introduction

Visual backward masking is a versatile tool for understanding the principles and limitations of visual information processing. In visual backward masking, a target is followed by a trailing mask. Temporal aspects, such as the stimulus duration and interstimulus interval between the target and the mask have been studied extensively, while largely neglecting spatial aspects (Breitmeyer & Ogmen, 2006). A few recent studies, however, have demonstrated that temporal and spatial factors interact strongly (Williams & Weisstein, 1984; Di Lollo, Enns, & Rensik, 2000; Cho & Francis, 2005; Duangudom, Francis, & Herzog, 2006; Dombrowe, Hermens, Francis, & Herzog, 2009). Particularly, the shine-through and the feature inheritance effects have revealed a number of complex spatio-temporal effects (Figure 1; e.g., Herzog & Koch, 2001; Herzog & Fahle, 2002; Herzog, Schmonsees, & Fahle, 2003). 

Figure 1

 

Shine-through and feature inheritance effect. (A) Shine-through is observed for regular gratings with seven or more elements. The target vernier appears to be superimposed onto the grating looking wider and brighter. The target vernier offset is clearly visible. (B) Feature inheritance is observed for gratings with less than seven elements. In this case the target vernier is largely invisible but its offset is expressed at the grating elements. Target vernier offset discrimination performance is worse for masks leading to feature inheritance compared those that give rise to shine-through (Herzog & Koch, 2001). (C) Inserting gaps in an extended grating leads to analogous effects as with the small five element grating.

Shine-through occurs when a target vernier is followed by a grating of seven or more elements. The target vernier appears to be brighter and superimposed on the grating (Figure 1A). In contrast, when the number of mask elements is less than seven, feature inheritance occurs (Figure 1B). In feature inheritance, the vernier is invisible while its offset is expressed at the grating elements, i.e., all lines appear to be offset in the direction of the vernier. Vernier offset discrimination in feature inheritance is worse compared to shine-through potentially because the vernier offset integrates with the aligned grating elements in feature inheritance but not in shine-through (Herzog & Koch, 2001). 

Not only the number of mask elements but also the spatial layout of the mask affects masking strength in the shine-through effect. For example, increasing the length of the mask elements that are spatially adjacent to the target vernier increases masking strength (Figure 2A-p1; Hermens & Herzog, 2007) and so does removing these elements, thus creating gaps (Figure 1C; Herzog & Koch, 2001). These two effects seem to be contradictory if mask energy were the only crucial factor determining masking strength. Increasing the length of some mask elements increases the overall energy in the mask while deleting elements leads to a reduction of overall mask energy. Still, both manipulations result in a strong increase in masking strength. We suggest these effects can be explained in terms of perceptual organization. For example, if two gaps are placed in a regular grating three separate gratings are perceived instead of one extended single grating. The smaller center grating leads to feature inheritance (Figure 1B). In the absence of irregularities, such as gaps or longer lines, all mask elements group together and shine-through occurs (Herzog, Fahle, & Koch, 2001; Herzog & Koch, 2001). 

Figure 2

 

Illustration of the masks used Experiment 1. (A) In the single, double length conditions, the position of the two longer elements was varied. (B) In the multiple conditions additional longer lines were added from the center outwards. (C) Sample masks for conditions in which either the luminance or the temporal delay were changed. All mask layouts shown in (A) and (B) were also presented with double luminance and a 20 ms delay. In Figure 2 double luminance is indicated by wider lines but in the actual display the lines had the same width.

Hermens and Herzog (2007) have shown that the global spatial structure rather than the local one determines masking strength. Performance deteriorates when the length of the two mask elements neighboring the vernier is doubled. When, however, the length of every second line is doubled, i.e., more number of longer elements are added, performance improves drastically relative to the single pair of long lines (Figure 2). These results cannot be explained by local interactions between the target and the mask elements next to it, because, locally, the neighboring lines are part of both masks. Moreover, these results cannot be explained by a range of possible accounts, including low-level filtering, Fourier analysis, theories of visual regularities, and overall mask energy (Hermens & Herzog, 2007). 

Here, we investigate how luminance and temporal manipulations of mask influence masking. 

General materials and methods

Participants

Data were obtained from paid, naïve students of the Ecole Polytechnique Fédérale de Lausanne (EPFL) and from two of the authors. The specific research question was not revealed to the naïve participants until the end of the experiment. All observers had normal or corrected-to-normal visual acuity as tested by means of the Freiburg visual acuity test (Bach, 1996). To participate in the experiments, observers had to reach a visual acuity of 1.0 (corresponding to 20/20) for at least one eye (except for one author who had an acuity of 0.9 only). The observers signed an informed consent form and were told that they could quit the experiment at any time. The experimental procedures were approved by the local ethics committee. 

Apparatus

Target vernier and mask elements with a luminance of about 80 cd/m² were displayed in the center of an otherwise dark Tektronix 608 X–Y-display equipped with a P11 phosphor. Stimulus presentation was controlled by a PC via fast 16 bit DA converters with 1 MHz pixel rate. The stimuli were refreshed at 200 Hz. Participants observed the stimuli from a distance of 2 m in a room illuminated dimly by a background light (approximately 0.5 lx). 

Procedure

The beginning of a new trial was marked by the appearance of a fixation cross and a small bar in the top of the screen that indicated the progress in the block. Participants were instructed to look at the fixation cross. On each trial, the target vernier was presented first and was immediately followed by a mask for 300 ms. After the participant's response, the screen remained blank for 200 ms before the beginning of the next trial. If participants did not respond within 3 s after stimulus offset, a beep sounded indicating a missed trial, and the trial was repeated at the end of the block. 

Each condition was tested in two blocks with 80 trials each. It took about 5 min to complete one block. In the first run, the sequence of conditions was randomized for each participant individually. To counteract effects of fatigue and practice in the averaged data, the order of blocks was reversed in the second run. The two measurements were collapsed into one mean. The standard mask was always presented in the first and the last block. The offset direction of the vernier target (left or right) was pseudorandomized across trials, restricting the number of subsequent same offset directions to four. 

Stimuli

Vernier stimuli were presented in the fovea and consisted of two vertical bars that were either vertically aligned or were slightly displaced in the horizontal direction (Figure 1). The segments of the target vernier and the mask elements were 10 arcmin long and separated by a vertical gap of 1 arcmin. Hence, the total length of the verniers was 21 arcmin. The spacing between all elements of the mask was 3.33 arcmin. 

For the target vernier, the direction of offset of the lower element with respect to the top element (left or right) was randomly selected on each trial. Both the top and the bottom segments of the vernier were offset in opposite directions with half the magnitude of the offset for a given trial. The vernier and the subsequently presented mask were centered on the middle of the screen. The target was presented for a duration of 20 ms (Experiment 1) or 30 ms for the rest of the experiments, while the mask was always shown for 300 ms. 

The rightmost mask in Figure 2A, with 25 homogeneous vertically aligned vernier elements, will be referred to as the standard mask. The other masks were created from the standard mask by changing the length, luminance, or temporal delay of certain grating elements. 

The different spatial arrangements are shown in Figure 2. The top row labeled single shows cases where only a single pair of elements was longer. The distance between the longer elements of the mask and the target vernier position increases from left to right in Figure 2A. In the second row (Figure 2B), labeled multiple, the pair of longer lines neighboring the target vernier was contained in all the masks. Masks were created by adding longer lines from the inside towards the outside. 

To refer to the masks, we will use a notation which reflects the position of the irregular elements. For example, if the elements at Position 1 on either side of the center are longer, the condition is referred to as p1. If Elements 1 and 3 are longer, the mask is called p1 + p3. “Every second” is used for the mask in which every second element is longer. All masks were symmetric around the center. 

Besides variations in the length of the mask elements, variations in luminance and time were used. Three luminance levels were tested: 160 cd/m² (double the standard luminance), 54 cd/m² (two-thirds), or 27 cd/m² (one-third). In addition, there was a gap condition where the irregularities in the mask were created with elements of zero luminance. In the temporal delay condition, a delay between 5 and 135 ms was used with respect to the rest of the mask element onsets (Figure 2C). 

For Experiments 3 and 4, the target vernier had a constant offset of 60 arcsec. Three types of masking gratings were used, all consisting of 25 elements. These 25 elements were either aligned (25N) as in Experiments 1 and 2 or offset by 10 arcsec. This offset was either in the same direction as the target vernier (25V) or in the opposite direction (25AV). See Figure 5 for examples. 

Task

The target vernier was randomly offset either to the left or to the right. Observers had to discriminate the offset direction by pressing one of two push buttons. In the first two experiments, offset discrimination thresholds of 75% correct responses were determined by using an adaptive staircase procedure (PEST; Taylor & Creelman, 1967) and fitting a cumulative Gaussian to the data using probit and likelihood analyses. We used the likelihood analysis only in cases where probit analyses failed to converge for numerical reasons. 

For the last two experiments, we determined how often the reported vernier offset was in accordance with the offset direction of the target vernier. Previous studies use the term “vernier dominance” for this measure (Herzog, Parish, Koch, & Fahle, 2003; Otto, Ögmen, & Herzog, 2006; Scharnowski, Hermens, & Herzog, 2007). 

Experiment 1: Spatial, luminance, and temporal irregularities

Experiment 1 investigates the effects of mask irregularities for the three basic visual dimensions: space, luminance, and time. 

Material and methods

Six observers, including one author, participated in the experiment. Masks with one type of irregularity, double length, double luminance, or a delay of 20 ms at one certain position were tested in one block. The vernier duration was 20 ms, mask duration 300 ms. 

The starting value for the offset of the target vernier was 150 arcsec. Offset discrimination thresholds were determined using an adaptive staircase. Auditory feedback was given for errors and missed trials. 

Results and discussion

Figure 3A shows the results for the double length condition. Vernier offset discrimination thresholds were higher for the p1 condition as compared to the standard condition (Figure 3A). Thresholds progressively decreased as the longer lines were positioned away from the center. In the multiple conditions, thresholds decreased as more pairs of longer lines were added. When every second line of the mask was longer, thresholds were almost comparable to that of the standard mask. A two-way repeated measures ANOVA revealed no main effect of condition, single or multiple; F(1, 5) = 1.61, p > 0.2, but a significant effect of the position of the outermost line, F(3, 15) = 11.8, p < 0.001. There was no significant interaction, F(3, 15) = 0.6, p > 0.5. Pairwise comparisons made for comparing local and global influences revealed significant differences between the p1 and the p1 + p3 + p5 masks, t(5) = 2.7, p < 0.05, and the p1 and every second conditions, t(5) = 4.1, p < 0.01. The double length condition replicates earlier findings by Hermens and Herzog (2007), demonstrating that results are consistent across studies. 

Figure 3

 

Mean thresholds for Experiment 1. (A) As in Hermens and Herzog (2007), masking strength decreases as the longer lines are placed further away from the center in the single conditions (red curve). In the multiple conditions (blue curve), adding more pairs of longer lines also decreases thresholds, even though the longer p1 elements were present in all multiple masks. Hence, the position of the outermost longer lines determines masking strength in both the conditions, rather than the p1 elements nearest to the target. (B) In the single, double luminance conditions, thresholds decrease as the double luminance lines are placed away from the center. In the multiple conditions, thresholds are fairly constant, indicating that for luminance, the two p1 lines determine masking strength. (C) In the temporal conditions, yet, another pattern of results is found. Thresholds for single pairs of flankers change nonmonotonically with the position of the delayed lines. In the multiple conditions, thresholds were constant. Moreover, the thresholds in the multiple conditions were lower than in the single ones, indicating lower masking strength. Error bars show the standard error of the mean for six observers. Thresholds for unmasked verniers are typically between 10–20 arcsec (Malania, Herzog, & Westheimer, 2007).

Figure 3B shows the results for the luminance conditions. Performance was strongly impaired compared to the standard mask when the mask elements next to the target vernier had a double luminance. As in the length conditions, vernier offset discrimination thresholds decreased as the double luminance lines were placed away from the center. However, when multiple double luminance lines were added, no drop in thresholds occurred contrary to the length condition. A two-way repeated measures ANOVA revealed no main effect of condition, single or multiple; F(1, 5) = 4.75, p > 0.08, and no significant effect of the position of the outermost line relative to the target, F(3, 15) = 2.25, p > 0.09. There was no significant interaction, F(3, 15) = 1.4, p > 0.3. Further analysis showed that the position of the outermost brighter line was significant in the single condition, F(3, 15) = 3.45, p < 0.05, but not in the multiple one, F(3, 15) = 1.13, p > 0.3. The performance for the multiple conditions was fairly constant across the different numbers of double luminance lines (for the every second condition, pairwise comparison with p1, t(5) = 1.4, p > 0.2, showing that the slight drop in performance that is apparent in the graph is not statistically significant). The results indicate that in the double luminance condition, the double luminance lines next to the target vernier determine performance rather than the overall structure. 

In the temporal conditions, yet another pattern of results was found. Here, performance in the single p1 conditions was better than for the intermediate (p3) and far away (p5) positions (Figure 3C). This nonmonotonic function of element position is quite different from the monotonic functions found in the length and luminance conditions. A two-way repeated measures ANOVA revealed a marginal main effect of condition, single or multiple, F(1, 5) = 5.55, p = 0.059, and a significant effect of the position of the outermost line relative to the target, F(3, 15) = 3.34, p < 0.05. There was also a significant interaction, F(3, 15) = 3.35, p < 0.05. Further analysis showed that the position of the outermost pair of delayed lines was significant in the single condition, F(3, 15) = 5.11, p < 0.05, but not in the multiple one, F(3, 15) = 0.54, p > 0.05. In the multiple temporal conditions, performance remained constant even after adding delayed elements to the single delayed lines next to the vernier. A lack of an effect of adding more irregular elements was also found in the luminance condition, but not in the length condition. 

Although, the overall thresholds in the temporal conditions were much lower than in the length and luminance conditions, it does not imply that temporal factors have weaker masking effects than length and luminance because parameters were chosen arbitrarily. For example, masking strength for masks created with a 45 ms delay might give rise to masking strength greater than that found for the length based masks (see Experiment 4). 

Mask irregularities related to the three basic visual dimensions yield highly different results. 

Experiment 2: Irregularities by luminance

In Experiment 1, the thresholds for all multiple luminance conditions were similar to the single p1 condition possibly because the global homogeneity of the every second mask may be counteracted by the increase in masking energy. In order to disentangle the influences of homogeneity and masking energy, we used masks with dimmer elements. 

Material and methods

We repeated the luminance conditions from the previous experiment with masks where the irregular element was two-thirds (54 cd/m²) or one-third (27 cd/m²) of the luminance of the standard elements (80 cd/m²). Moreover, there was a gap condition where the irregularities in the mask were created with elements of zero luminance (or no contrast with respect to the background). The data was collected in three subexperiments, one for each luminance level. Four observers, three new and one author, participated in each subexperiment. 

The display duration of the target vernier was calibrated to obtain a clear shine-though effect for the standard mask for each observer individually. To meet this criterion, we used a vernier duration of 30 ms instead of the 20 ms used in Experiment 1. 

Results and discussion

Figure 4 shows the data for Experiment 2 along with the data for double luminance (Figure 4A) replotted from the Experiment 1. Phenomenologically, the 54 cd/m² lines appeared very similar to the standard luminance elements. Thresholds obtained were also similar to that for the standard 25 element mask. 

Figure 4

 

Mean thresholds for Experiment 2. (A) Single and multiple double luminance conditions replotted from Figure 3B. (NOTE: the range of the y-axis is different for A compared to B–D). (B) In the 54 cd/m² condition, all thresholds were similar to the ones in the standard (std) condition and much smaller than in the other luminance conditions. (C) In the single 27 cd/m² luminance conditions, thresholds decrease as the dimmer lines are shifted away from the center. In the multiple conditions, adding more pairs of dimmer lines also decreases thresholds. These results are similar to the length condition in Figure 3A and indicate that the position of the outermost dimmer lines determines masking strength, rather than the p1 elements. (D) In the gap (0 cd/m²) conditions, thresholds for single gaps change nonmonotonically with the position of the gap for both the single and the multiple conditions. This pattern of results indicates a greater influence of the p1 elements. The nonmonotonic pattern for the single condition is qualitatively similar to the temporal delay based masks in Experiment 1. Interestingly, for all luminance based masks, the thresholds for the multiple condition were higher than the single condition both for brighter and dimmer irregularities.

Figure 5

 

Percent vernier dominance for of Experiment 3. The sequence on the extreme right shows the time course of each trial, with a target vernier followed by a mask. The 25 elements of the mask were either aligned (25N) or offset by 10 arcsec in the same direction as the target vernier (25V) or in the opposite direction (25AV). The influence of the mask increased when delays increased from 5 ms to 135 ms (Subplots A to D). The difference between the pro-offset (V) and the anti-offset (AV) conditions reflects integration of mask and vernier offset. Strongest integration occurred for the 45 ms and 135 ms conditions. Unspecific masking is indicated by an overall decrease in vernier dominance for all three offset conditions. Masking is particularly strong in the 135 ms condition. There is no significant difference between the data points for the three std mask conditions (i.e., masks without a delay) for all four subplots (A–D). Note: For the purpose of illustration, only a subset of the 25 mask elements is shown in the illustrations. The vernier dominance for the unmasked target vernier was 99.4% (not shown).

For the 27 cd/m² luminance condition, the thresholds were highest for the p1 condition and decreased progressively as the dimmer lines moved away from the center (Figure 4C). In the multiple conditions, thresholds decreased as more pairs of dimmer lines were added to the mask. There was a significant effect of position of the outermost line, F(3, 9) = 6.92, p < 0.01. Further analysis showed that the effect of the position of the outermost dimmer line was significant in the single, F(3, 9) = 6.15, p < 0.01, and in the multiple conditions, F(3, 9) = 4.13, p < 0.05. When every second line was of lower luminance, performance was significantly different from the p1 mask (for the every second condition, pairwise comparison with p1, t(3) = 3.05, p < 0.05). The lack of a statistically significant difference between the single and the multiple conditions indicates that the global configuration of the dimmer elements determines the masking strength, similar to the length conditions in Experiment 1. 

For the gap condition, a nonmonotonic function of element position was observed (Figure 4D). This is quite different from the monotonic functions found in the other luminance conditions but qualitatively similar to the temporal delay based masks in Experiment 1. A two-way repeated measures ANOVA revealed no main effect of condition, single or multiple; F(1, 3) = 6.27, p > 0.08, and no significant effect of the position of the outermost line, F(3, 9) = 3.3, p > 0.07. However, there was a significant interaction, F(3, 9) = 4.2, p < 0.05. Further analysis showed that the position of the outermost gap was significant in the single, F(3, 9) = 5.28, p < 0.05, but not in the multiple condition, F(3, 9) = 1.29, p > 0.33. This pattern is qualitatively similar to the 160 cd/m² condition and indicates that only the elements neighboring the vernier determine masking strength. 

One interpretation of these results is that lower masking energy for the 27 cd/m² mask compared to the 160 cd/m² or the 54 cd/m² makes the effect of homogeneity of multiple irregular elements more evident. However, this does not explain why there is no such effect of homogeneity for the gap condition with even lower masking energy. Interestingly, for all luminance based masks, the thresholds for the multiple conditions were higher than in the single conditions. Since this is observed for both brighter and dimmer irregularities, the masking energy of the mask per se fails to explain the results reported here. 

Experiment 3: Irregularity by delay

In the single delayed conditions, observers reported that the center elements of the masking grating appeared to be offset even though elements were, in fact, aligned, suggesting feature inheritance effects. Experiment 3 investigates this effect in more detail. 

Material and methods

Five observers, including one author, participated. We used delays of 5 ms, 15 ms, 45 ms, and 135 ms. The vernier duration was 30 ms, the mask duration 300 ms. The offset size of the target vernier was set at 1 arcmin. The task was to judge the perceived direction of offset of the target vernier. Before the experiments, participants received 80 practice trials. If during the practice trials, the participants reported that they could not see the target vernier, they were instructed to “use their best judgment for the direction of offset,” the same strategy they were asked to also use during the main experiment. Performance was quantified as the percentage of responses in which the perceived offset corresponds to the actual direction of the offset of the target vernier (vernier dominance). Higher values (>50%) indicate that the target vernier dominated the performance while lower values (<50%) indicate that the offset of elements of the masking grating contributed more to the performance. The multiple conditions of Experiments 1 and 2 were not used in this experiment. 

To test the interaction between mask and target vernier offsets, the 25 mask elements were either aligned (25N), as in the previous two experiments, or offset by 10 arcsec (i.e., six times smaller than the 1 arcmin offset of the target vernier). This offset was either in the same direction as the target vernier (25V) or in the opposite direction (25AV). Conditions were presented blockwise with V and AV masks in one block and N masks in a different block. In contrast to Experiments 1 and 2, no feedback was provided. 

Results and discussion

For all masks irrespective of the offset of mask elements, if the delay segregates the five center elements from the peripheral elements, we expect feature inheritance to occur. If the offsets of the mask elements completely dominate performance, we expect vernier dominance to be below 50% for masks with anti-offset elements. For the 25 elements of the mask, we expect shine-though to occur, irrespective of the offset direction of the mask elements. This would lead to a higher value of vernier dominance even for the mask with anti-offset elements (25AV). 

When there was no delay of the mask elements, the results for the 25V, 25N, and 25AV standard conditions show no significant difference in vernier dominance (rightmost data points for all subplots in Figure 5). Hence, the offsets in the 25V, 25N, and 25AV standard masks do not integrate with the vernier offset indicating that shine-through is not influenced by the mask offsets (see also Hermens, Scharnowski, & Herzog, 2009). 

For masks with a 5 ms delay of the p1 or p3 lines, dominance was fairly constant for all three mask types (Figure 5A). The results, which were similar to the standard masks, indicate the presence of shine-through effect for all mask offsets in the 5 ms delay condition. However, the pattern of results changed for masks with longer delays. 

As shown in Figure 5B, for masks with a 15 ms delay of the p1 or p3 lines, there was a significant effect of the offset direction of the mask elements on vernier dominance, 25V, 25N, or 25AV, F(2, 8) = 6.33, p < 0.05, and a significant effect of the position of the delayed line relative to the target, p1, p3, or standard, F(2, 8) = 10.34, p < 0.01. This effect increased with longer delays (Figures 5C and D). For masks with a 45 ms delay ANOVA shows the following factors as significant: offset of mask elements, 25V, 25N, or 25AV, F(2, 8) = 7.21, p < 0.05; position of delayed line, p1, p3, or standard, F(2, 8) = 85.71, p < 0.001, and a marginally significant interaction, F(4, 16) = 2.98, p = 0.051) For masks with a 135 ms delay, 25V, 25N, or 25AV, F(2, 8) = 12.27, p < 0.01; p1, p3, or standard, F(2, 8) = 51.19, p < 0.001. There was significant interaction, F(4, 16) = 4.20, p < 0.05. 

Thus, for delays longer than 15 ms, there seems to be a significant interaction between the offset of the target vernier and the mask elements. There seem to be two components for this interaction: first, an unspecific type of masking and second, feature inheritance. The unspecific type of masking is reflected by lower vernier dominance values for the masks consisting of pro-offset (25V) and aligned verniers (25N). This effect increases with the delay. Interestingly, the masking is by far strongest in the p3 compared to the p1 condition even though the delayed lines are not neighboring the vernier target in the p3 condition. This suggests a feature inheritance effect which predominantly occurs for five element gratings (e.g., Herzog et al., 2003). Our data suggest that a delay as short as 45 ms suffices to break the 25 element mask into smaller masks. Similar fast grouping effects were found previously (Herzog, Fahle, & Koch, 2001; Herzog et al., 2008; Hermens et al., 2009). 

Previous studies have demonstrated that in feature inheritance the vernier and the mask offsets are irretrievable integrated (Scharnowski, Hermens, & Herzog, 2007; Hermens, Scharnowski, & Herzog, 2009; Scharnowski, Rüter, Jolij, Hermens, Kammer, & Herzog, 2009; Hermens, Scharnowski, & Herzog, 2010). We suggest that similar inheritance occurred here. Hence, we argue that the differences between the conditions are a consequence of a perceptual bias that depends on the temporal delay of the irregular elements. 

Experiment 4: Feature integration for spatial and luminance irregularities

In Experiment 3, we showed that vernier and mask offsets integrate when a delay segregates the homogeneous 25 element grating into smaller gratings. In Experiment 4, we investigate whether such integration also takes place for spatial or luminance manipulations. 

Material and methods

The same participants took part as in Experiment 3. The vernier duration was 30 ms, the mask duration 300 ms, and the offset size of the target vernier was 1 arcmin. The elements of the mask were either aligned (25N) or offset either by 10 arcsec in the same direction (25V) or in the opposite direction (25AV) as the vernier offset (Figure 6A). The p1 and p3 elements were either double as long (space) or double as bright (luminance). The conditions were presented blockwise and no feedback was given. 

Figure 6

 

Stimulus sequences and vernier dominance for Experiment 4. (A) The 25 masks elements were either aligned (25N) or offset by 10 arcsec in the same direction as the target vernier (25V) or in the opposite direction (25AV) (Only seven inner elements out of the 25 elements are shown in Figure 6 for the sake of clarity). (B) In the double length conditions, vernier dominance increased as the position of the longer lines moved away from the center. The separation between the performance in the same (25V), neutral (25N), and anti-offset (25AV) conditions indicates feature inheritance. Performance for masks with double luminance elements was similar to the double length conditions. In the delay conditions, performance varied nonmonotonically as discussed in the previous experiment. The data in the standard 25N, 25V, and 25AV masks and the delay conditions are replotted from Figure 5.

Results and discussion

The data for the 25V, 25N, and 25AV standard masks are replotted from Experiment 3. In contrast to the standard masks, there were significant differences for the different mask offset directions (25V, 25AV, 25N) when the mask contained irregular elements. Vernier dominance was lower for the 25AV masks and higher for the 25V masks compared to the neutral 25N mask. Moreover, the position of the irregularity played a role. For both the length and the luminance conditions, dominance increased monotonically from the p1 to the p3 and was highest for the standard conditions. This monotonic trend was largely independent of the offset of the mask elements. 

For masks with a double length for the p1 or p3 lines, there was a significant main effect of mask offset, 25V, 25N, or 25AV, F(2, 8) = 5.54, p < 0.05, and a significant effect of the position of the longer line, p1, p3, or standard, F(2, 8) = 17.77, p < 0.001. There was no significant interaction, F(4, 16) = 1.00, p > 0.43. 

For masks with double bright p1 or p3 lines, there was no significant main effect of mask offset, 25V, 25N, or 25AV, F(2, 8) = 4.15, p > 0.058, but a significant effect of the position of the brighter line relative to the target, p1, p3, or standard, F(2, 8) = 11.25, p < 0.01. There was no significant interaction, F(4, 16) = 1.67, p > 0.20. 

In contrast with length and luminance, for the delay conditions, performance varied nonmonotonically with the position of the irregular elements as already mentioned in the previous experiment and there was a significant interaction between offset and position of the irregularity. 

Evidence for shine-through is found for the homogeneous 25 element gratings and is independent of the direction of the offset of the mask elements (i.e., for the 25V, 25N, and 25AV gratings). When, however, irregularities within the grating break the homogenous structure, feature inheritance takes place as demonstrated by a lower vernier dominance for anti-offset and higher dominance for the pro-offset mask elements compared to the neutral mask. The smaller subunits, created by the irregularities, also mask the vernier more strongly than the large 25 elements grating. 

Hence, performance was qualitatively different in these conditions compared to the delay conditions. 

Discussion

The shine-through effect reveals complex spatiotemporal characteristics. Small, local changes in the mask layout, e.g., increasing the length of mask elements, can strongly change the masking strength (Herzog & Koch, 2001; Herzog & Fahle, 2002). Hermens and Herzog (2007) showed that increases in masking strength by local changes can be counteracted by global changes in the mask. When the length of every second mask element was doubled, performance improved compared to when only the length of two elements neighboring the vernier was doubled (Figure 2), even while the local elements in the mask adjacent to the target remained the same. We suggested that the addition of longer lines in a regular fashion leads to a homogeneous mask leading to shine-through effect. 

Here we studied whether such effects of global layout of the mask occur for other basic visual dimensions. Mask irregularities were created by using temporal (delayed elements) and luminance (brighter and dimmer elements) variations. The results in the temporal and luminance conditions strongly deviate from those found for length irregularities. While doubling the p1 elements' luminance led to similar effects as increasing their length, adding further double luminance lines did not improve performance (Figure 3). Hence, with luminance, masking strength is determined by the p1 double luminance elements. However, adding more double luminance lines also increases mask energy. We therefore examined the influence of adding dimmer elements to the mask with similar results. It seems that mask energy (duration times luminance) per se cannot explain masking strength because increases as well as decreases of luminance led to increased masking strength (Figure 4). 

As with luminance, adding more delayed elements leaves performance roughly unchanged. It seems that local interactions determine performance. These results are interesting with respect to theories on temporal binding. It has been shown that temporal asynchronies can bind together spatially distinct elements (Fahle, 1993; Lee & Blake, 1999; Farid & Adelson, 2001; Kandil & Fahle, 2001; Farid, 2002; Blake & Lee, 2005). Importantly, it was shown that one frame of asynchrony at the beginning of the stimulus presentation is sufficient for the grouping (Beaudot, 2002; Bex & Dakin, 2002). Hence, in the every second condition, all delayed elements may have been bound together by synchrony leading to reduced interference from the p1 mask elements. However, we found performance in the p1 condition for the delayed elements to be very comparable to the multiple conditions. Even more interestingly, performance did not improve in the single delayed element conditions when the pair of lines moved further away from the target vernier. Instead of binding of mask elements, we found binding of the vernier offset with grating elements, i.e., inheritance-like effects (Experiment 3). Possibly, the delay led to a grouping of the lines between the delayed elements. It should be mentioned that the delays per se are hardly visible. 

Whereas the basic shine-through conditions could well be explained by dynamic local inhibitory and excitatory neural interactions in a Wilson-Cowan type of network (Wilson & Cowan, 1973; Hermens, Luksys, Gerstner, Herzog, & Ernst, 2008; Hermens, Scharnowski, & Herzog, 2009, 2010), the present data suggest other processes in the current experiments (see Appendix for details). Specifically, local dynamic interactions fail to explain the strong improvement in performance when additional longer elements are added to the mask (Experiment 1). Likewise, local neural interactions fail to explain why adding additional reduced luminance elements to the mask improves performance (Experiment 2), and cannot provide an explanation for the feature inheritance effects (Experiments 3 and 4). In contrast, the Wilson-Cowan type model does explain the effects of double luminance elements, single double length elements, and mask elements with a temporal delay (Experiment 1). For the remainder of the effects, processes that operate more globally appear to be at work. 

Our study adds to the recent trend of combining research on spatial and temporal aspects of visual perception. Up to recently, only a few studies on visual masking had investigated complex spatial aspects (Werner, 1935, Williams & Weisstein, 1984, Li, Thier, & Wehrhan, 2000, Cho & Francis, 2005; Francis & Cho, 2005). Even if studies examined spatial aspects, they often focused on rather basic spatial aspects (Alpern, 1953; Kolers, 1962; Sekuler, 1965; Sturr, Frumkes, & Veneruso, 1965; Sturr & Frumkes, 1968; Parlee, 1969; Growney, 1977; Bridgeman & Leff, 1979; Wehrhahn, Li, & Westheimer, 1996). The present work adds to the evidence that such an approach is not feasible and that spatial and temporal aspects of visual perception should be studied together (Herzog, 2007). Importantly, the present findings show that global aspects of the mask layout can play an important role strongly depending on the type of manipulation i.e., spatial, luminance, or temporal. 

Acknowledgments

This research was in part supported by a Marie Curie Career Integration Grant (#293901) from the European Union awarded to Tandra Ghose. We also thank two anonymous reviewers for their comments and criticisms. 

Commercial relationships: none. 

Corresponding author: Tandra Ghose. 

Email: [e-mail] 

Address: University of Kaiserslautern, Kaiserslautern, Germany. 

Appendix: Simulations with a Wilson-Cowan type model

Introduction

In addition to collecting behavioral data for the different experimental conditions, we also performed simulations using a neural network model applying dynamic local inhibitory and excitatory interactions. The model that we used was inspired by the work by Wilson and Cowan (1973) and we therefore refer to it as the Wilson-Cowan type model (Herzog et al., 2003). Previous simulations with the model have shown that, despite the relative simplicity of the model (Francis, 2009), the model can account for a large range of findings in visual masking (Hermens et al., 2008) and feature fusion (Hermens, Scharnowski, & Herzog, 2009, 2010). For example, the model can well explain why breaking the regularity of a 25 element grating mask (Figure 1, left) by inserting two gaps in the grating (i.e., removing two elements from the grating; Figure 1, right) leads to a strong deterioration in target performance (Hermens & Ernst, 2007). Because of the structure of the model, involving two layers of neurons (one inhibitory layer and one excitatory layer), edges of regularly spaced inputs are highlighted, whereas the inside of the regular structures receives less neural activation. It is this highlighted activation from the mask that interacts with the signal of the target. This means that if the highlighted elements are far away from the target, interference is weak, resulting in good target based performance. If the highlighted elements are near the target, interference is strong, and performance on the target will be poor. By inserting two gaps in the grating, additional highlighted edges near the target position are formed (at the edges of the gaps), which interfere with the target signal, explaining why performance is weak for a mask containing two gaps. 

Whereas the highlighting of edges explains the effects of two gaps in a grating well, it is unclear whether the same holds for irregularities formed by increasing the length of mask elements. To examine the performance of the model for other types of mask irregularities than gaps, we conducted simulations with the stimuli from the present study. In these simulations, we kept the structure and parameters of the earlier model. If we can demonstrate the new set of findings can be explained with the same model as used before, this would provide evidence for a common mechanism underlying the old and the new findings. Otherwise, we must conclude that other mechanisms are at work, requiring further model development. 

Methods

Details about the model can be found elsewhere (Hermens et al., 2008; Hermens, Scharnowski, & Herzog, 2009, 2010), so we here describe only the general structure of the model. 

The model consists of two layers of neurons. One layer contains neurons that inhibit the activity of other neurons, both within the same layer as well as in the other (excitatory) layer. The second layer consists of neurons that excite other neurons, both within the same layer, as well in the other (inhibitory) layer. In addition, activity of all of the neurons is subject to decay over time. The input to the model consists of the same sequence of stimuli as presented to the participants in the experiment. Input enters both layers simultaneously and consists of a two dimensional (2D) representation (across time) of the visual stimuli, coded as intensities. Pixels belonging to the target vernier or the mask have intensity equal to one, whereas the background intensity is set to zero. Elements of double luminance have an intensity of two. 

Whereas it is possible to examine the network's response across the entire stimulus presentation interval, simulations suggest that most of the neural interactions occur within the first 60 ms, after which the activity in the model merely decays. To limit the amount of time required for the simulations and to reduce noise due to a decaying trace of the stimuli, the read-out time was set at 60 ms after the onset of the target vernier as in earlier simulations. At read-out, the activity in the excitatory layer is compared to a template of the target vernier or the oppositely offset vernier (antivernier). This value is converted to offset discrimination thresholds (Experiments 1 and 2) or the percentage of trials according to the vernier offset (vernier dominance; Experiments 3 and 4) using the linking hypotheses introduced earlier (for thresholds: Hermens et al., 2008; for percentages: Hermens, Scharnowski, & Herzog, 2009). 

Results

Figure A1 shows the predicted vernier discrimination thresholds and vernier dominance values for each of the experiments. The model explains the experimental data on some points, but clearly fails on others. For example, for Experiment 1, the model correctly predicts higher thresholds for the single than for the multiple lengths and luminance conditions, whereas for the time condition, the opposite relationship is found. However, the model fails to predict the strong decrease in thresholds when adding more double length lines to the mask. Interestingly, the model correctly predicts the nonmonotonic function for the temporal conditions. Similarly, the data for Experiments 3 and 4 are explained by the model on only some conditions. The model correctly predicts that anti-offsets in the mask lead to lower vernier dominance and that the difference between the anti-offset and neutral vernier offset grows when the delay increases in duration. It fails, however, to predict that vernier dominance is lower for the p3 than for the p1 conditions in the temporal conditions. 

Figure A1

 

Predictions from the Wilson-Cowan type model for each of the experiments. For Experiments 1 and 2 the model predicts the pattern of results for the double luminance and time conditions fairly well. For Experiments 3 and 4, the model correctly predicts that vernier dominance will be lower for the anti-offset than for the neutral and same-offset mask elements. However, the effects of the position of the irregularities are not well predicted.

Conclusion

Whereas the model explains certain aspects of the experimental findings, it fails to explain others. This suggests that local inhibitory and excitatory interactions do not suffice to explain the present results and that other processes, possibly operating at a higher level and more global mode of processing, are involved.