Evaluating integration of letter fragments through contrast and spatially targeted masking

Abstract

Four experiments were conducted to gain a better understanding of the visual mechanisms related to how integration of partial shape cues provides for recognition of the full shape. In each experiment, letters formed as outline contours were displayed as a sequence of adjacent segments (fragments), each visible during a 17-ms time frame. The first experiment varied the contrast of the fragments. There were substantial individual differences in contrast sensitivity, so stimulus displays in the masking experiments that followed were calibrated to the sensitivity of each participant. Masks were displayed either as patterns that filled the entire screen (full field) or as successive strips that were sliced from the pattern, each strip lying across the location of the letter fragment that had been shown a moment before. Contrast of masks were varied to be lighter or darker than the letter fragments. Full-field masks, whether light or dark, provided relatively little impairment of recognition, as was the case for mask strips that were lighter than the letter fragments. However, dark strip masks proved to be very effective, with the degree of recognition impairment becoming larger as mask contrast was increased. A final experiment found the strip masks to be most effective when they overlapped the location where the letter fragments had been shown a moment before. They became progressively less effective with increased spatial separation from that location. Results are discussed with extensive reference to potential brain mechanisms for integrating shape cues.

Introduction

Human vision is capable of a great number of skills, including the ability to identify shapes that have been integrated from incomplete parts. Visual persistence plays a central role in providing this ability, presumably by sustaining the activity of neurons that register the elemental shape cues until they can be combined by cortical mechanisms (Averbach & Coriell, 1961; Coltheart, 1980; Greene & Visani, 2015; Sperling, 1967). It is likely that the lines and edges of a given shape are processed by orientation-selective neurons and integrated farther down the visual stream into a coherent message (Hubel & Wiesel, 1959; Hubel & Wiesel, 1968; Kreiter & Singer, 1992; Snodderly & Gur, 1995). It has been shown that these neurons can synchronize within the millisecond range and provide linkage of signals over extended time intervals (Engel, König, Kreiter, Schillen, & Singer, 1992; Fries, Neuenschwander, Engel, Goebel, & Singer, 2001; König, Engel, & Singer, 1995; Singer & Gray, 1995). To understand the nature of these mechanisms more fully, we conducted four experiments, requiring participants to identify letters that were sequentially displayed as contour fragments, with manipulation of contrast polarity as well as several masking conditions. 

Visual masking has been extensively used to explore perceptual mechanisms (Ogmen, Breitmeyer, & Melvin, 2003; Sperling, 1963; Turvey, 1973). Masking might impair shape recognition by creating a substitute or an updated version of the target that is an inaccurate model of the target itself (Enns & DiLollo, 2000; Goodhew, 2017; Kahneman, Treisman, & Gibbs, 1992). This may be reflected by work that has shown masking interferes with a neural signal of the target that comes into visual awareness prior to recognition (Koivisto & Revonsuo, 2010). However, most masking paradigms use a full target image rather than targets broken into multiple fragments. It is unclear whether masking prevents sufficient information held in memory from being integrated. Our experiments were designed to clarify how masking affected the integration process. 

Early work on primary visual cortex found that there are contrast-sensitive cells that react only to light or dark bars (Celebrini, Thorpe, Trotter, & Imbert, 1993; Hubel & Wiesel, 1959; Sceniak, Ringach, Hawken, & Shapley, 1999). This finding has been used to explain why masking is more effective when the mask matches the polarity of a target (Arnold & Anstis, 1993; Becker & Anstis, 2004). However, it is less clear whether this relationship holds when both the target and mask are flashed multiple times. This repetition provides an additional challenge of keeping the mask and target separate while still integrating the necessary fragments of the target stimulus. Our results give insight into the role of contrast and polarity over longer periods of integration. 

It has been further suggested that neural signals carry vital information about spatial location that can be classified as local and global signals (Averbach & Coriell, 1961; Kooi, Toet, Tripathy, & Levi, 1994). The role of cue location has been demonstrated in crowding studies, in which spatial distance of distractors from a target has a significant role in impairing recognition of the target (Bouma, 1970; Leat, Li, & Epp, 1999). Work on transient and sustained channels has shown that they are differentially activated by changes in spatial location (Tolhurst, 1975). To address this issue, our study compared two types of masks: one in which the spatial location of the mask changes with the target and one in which the mask stays static. 

The first type of mask uses a common metacontrast masking paradigm, in which the mask does not overlap with the target (Enns & DiLollo, 2000; Kolers & Rosner, 1960; Turvey, 1973). The second uses a full-field mask, in which the mask covers the entire screen and appears after a target is displayed (Lamme, Zipser, & Spekreijse, 2002; Turvey, 1973). Both have been shown to be effective but have been theorized to disrupt different recognition processes in the visual system (Enns & DiLollo, 2000). Our experiment compared the effect of metacontrast masking and full-field masking on recognition of a target with changing locations. We theorize that integration is disrupted more heavily when the spatial location of the mask changes with the target compared with masks that rely more on a repeated, extended, global code. 

Through a series of four experiments, we used incomplete letters to explore the role of integration on recognition. The first experiment found that letters can be integrated when displayed as successive fragments at very low contrast levels. The second used strips of masks that follow each letter fragment at different contrast levels and polarities. The third tested the role of global and local cues using a full-field mask rather than a strip mask. The fourth examined the role of spatial codes (Kooi et al., 1994) using strip masks that were displayed at various distances relative to the target letter fragments. 

Methods

Institutional approval and participants

Protocols for the research project were approved by the University of Southern California (USC) Institutional Review Board. Eighteen USC undergraduates, six males and 12 females, served as participants. Ages ranged from 18 to 29 years. The nature of the task was explained to each participant, and each was informed of their ability to terminate the experiment at any time, without penalty. Each signed a consent form. 

Equipment, ambient lighting, and display measures

Experiments were conducted in a small test room with normal fluorescent ceiling lights, which provided 26 lux of ambient illumination. Visual stimuli were displayed on a liquid-crystal display (LCD) monitor (LG Electronics, Seoul, South Korea) with a screen size of 36 cm by 61 cm, that was positioned 2 meters in front of the participant. Experimental protocols were controlled by custom applications written in C++ and executed on a Mac Mini (Apple, Cupertino, CA). 

The LCD display provided 256 levels of brightness (monitor units), perceived as black at 0 and white at 255. The monitor was configured at low intensity so that displays were close to threshold perception levels. A color calibrator device was used to measure absolute emission in cd/m² corresponding to the unsigned integer monitor units (see Figure 1). Full brightness (255 monitor units) was measured as 44 cd/m². 

Figure 1.

 

Measures of monitor emission provided a gamma curve that reflects image intensity (cd/m²) as a function of monitor units.

For each of the experiments, the background intensity was 64 monitor units (dark gray) or 1.87 cd/m², and the contrast of letter fragments varied as negative (dark) departures from background, as specified for each experiment below. “Contrast” refers to differences in monitor units, except where “Weber contrast” or ratios of true luminance (cd/m²), interpolated from the calibrator measurements, are specified. For example, a contrast of +50 monitor units above the 64 background level corresponds to 6.045 cd/m² for the stimulus and 1.87 cd/m² for the background, this being a Weber contrast of (6.045 – 1.87)/1.87 = 2.23. 

Letter displays

All 26 letters of the English (Latin) alphabet were used in these experiments, with a font style of Arial 60 Outline. The use of outline strokes and delivery through multiple fragments increased the complexity of contour relationships. Letters were displayed at half the height of the monitor screen, as illustrated in Figure 2A. Letter height at the viewing distance of 2 meters was 4.2 degrees of visual angle (hereafter abbreviated as arc°). Note that this measure, as well as others provided in the Methods section, are specified with decimal precision; however, they are rounded to simplify the discourse in other sections of the report. Letters were displayed upright (i.e., without tilts) at eccentricities that were 2.1 arc° from center and at an angle that was randomly chosen on each trial of each experiment. The eccentricity served to inhibit recognition based on distinctive spatial attributes, such as having a vertical stroke in the center (I and T) or being to the left or right of center (L and J). 

Figure 2.

 

Each panel represents the full screen of the display monitor. (A) Each trial of each experiment displayed a randomly chosen letter at a location that was eccentric to the screen center. The alternative locations for letter display are indicated by the blue circle. (B) Dashed lines illustrate how the screen image was partitioned into 20 strips, with strip orientation being chosen at random on each trial. Each strip was displayed one at a time, providing a brief image of letter fragments as the sequence passed across the letter. Each panel illustrates the display of one strip within the sequence, showing the letter fragments that would be visible for a moment before the strip sequence moved on. For the purpose of illustration, contrast levels used for these illustrations are much higher than those used in the experiments, which were performed under dark and low-contrast conditions.

Figure 2B illustrates that the screen image on a given trial was functionally partitioned into 20 strips. This cut the letter contours into fragment subsets that would be displayed sequentially, as detailed below. The orientation of the strips was determined at random for a given trial. For non-vertical orientations, the number of strips required to display all letter fragments depended on the orientation that was chosen and the width of the letter being displayed. The number of strips that could contain letter fragments ranged from two to 12, with 79% of the letters requiring four to six strips. Across all orientations, the mean strip width was 0.71 arc°, and the mean number of strips displaying fragment subsets was 5.9. Both median and mode were six strips, which is the number used for reporting results of experiments and for illustrating letter-fragment displays. 

Each of the strips illustrated in Figure 2 is designated as a “letter strip” even if the strip is displaying background luminance (i.e., does not contain any letter fragments). This is to distinguish the stimuli being displayed by these strips from what was delivered by “mask strips,” described below. 

Mask displays

Figure 3 provides examples of mask patterns that were provided in three of the following experiments. Mask patterns were constructed by cutting letters into four pieces, designated as “mask pattern elements,” which were quasi-randomly positioned in relation to one another. The density of mask patterns was controlled by adjusting the spacing among the pattern elements using the density of letter pixels as the standard. Average letter density was defined as the percent of letter-forming pixels per average bounding rectangle, this being the rectangle that enclosed the pixels of a given letter. This was calculated to be 14%. Mask patterns therefore used 14% of the pixels for display of a given pattern, matching the density of letters. 

Figure 3.

 

Mask patterns were constructed to have a pixel density that matched the pixels per unit field area as letters. (A) The two panels show that the mask patterns could be dark or light. The full-field patterns were used for masking in Experiment 3. (B) The mask patterns could be cut into strips that were at the same orientation as letter strips and overlapped the same areas. These mask patterns being displayed by a strip could be dark or light in Experiment 2, and dark mask strips were used in Experiment 4.

The masks could be used as a “full-field” pattern, meaning that it filled the entire display screen, as shown in the upper panels of Figure 3. Alternatively, the lower panels illustrate that the pattern could be cut into 20 adjacent strips that overlapped the locations of letter strips. In other words, for every image zone being displayed by a letter strip, a mask strip was provided at the same orientation and positioning, thus covering the same zone. These have been designated as “mask strips.” 

Task administration

Participants were tested individually in sessions that each lasted less than an hour. Each participant was first told about the kind of displays that would be judged and were informed of their ability to discontinue testing at any time and for any reason (or no reason). Each signed a consent form and were then seated facing the display monitor. The experimenter sat about 6 feet away, controlling display trials with a separate monitor and keyboard that could not be seen by the participant. 

To begin an experimental session, a fixation point was provided at the center of the monitor screen that was otherwise blank (i.e., emitting at the background luminance of 64 monitor units). The experimenter launched a given trial by pressing an on-screen button, delivering the frame sequence that included the successive letter fragments (as well as mask patterns in Experiments 2 to 4). The fixation point reappeared after the full stimulus sequence had been displayed and remained on until the launch of the next trial. 

The participant was expected to name the letter and was asked to guess if they were not sure which letter had been shown. The experimenter recorded the response by selecting that letter from an on-screen keyboard. The computer logged information about display conditions for that trial, what response was rendered by the participant, and whether that response was correct. The experimenter was not provided with information on what letter had been displayed and thus had no basis for knowing whether or not the response was correct. After the response was entered into the computer, the experimenter pressed an on-screen button to deliver the next trial. This continued until all treatment conditions had been displayed for a specified number of times, requiring also that the same number of trials be provided for each experimental condition. 

Experiment 1

The Experiment 1 protocol provided a common task that was administered to every participant across this series of experiments. This protocol provided a simple progression of letter strips, varying contrast of the letter fragments to derive a probability of recognition model for each of the participants. Previous work found that such briefly displayed sequences of letter fragments can be integrated, allowing recognition of the letter from which the fragments were derived (Zhang, Morrison, Wang, & Greene, 2022). Display of the 20 strips was controlled by custom software, such that the image content of a given strip was delivered to the LCD display during a single time frame. Therefore, the contents of a given letter strip replaced the uniform background that filled the screen for just a moment, providing a brief glimpse of any letter fragments that might be present or providing no perceptible change in the image if the strip displayed only background. An illustration of the Experiment 1 task is provided in Figure 4. 

Figure 4.

 

The Experiment 1 protocol provided a sequence of time frames that successively displayed adjacent fragments of the letter to be identified. Here, we have illustrated the average number of frames for display of a given letter, each providing a brief glimpse of the shape cues. Time frames that would display only background luminance (i.e., those coming before or after the letter) are not shown in the illustration.

Figure 4 shows six fragment subsets, reflecting the median number of strips crossing all 26 letters at random strip orientations. At 16.7 ms for display of each strip, the time for display of the average letter was 100 ms. With mean letter size being just over 4 arc°, travel speed of the sequence was roughly 40 arc° per second. This is far faster than the travel speeds of slits that have been used in the study of shape-encoding mechanisms (e.g., Bognar & Vogels, 2021; Orlov & Zohary, 2018; Rock, 1981). How sequence velocity and other factors affect integration of the fragments will be discussed subsequently. 

For Experiment 1, as well as those that followed, the letter to be used on a given trial was chosen at random, without replacement, with the full 26-letter inventory being restored once each had been shown. The orientation of strips varied at random from one trial to the next, being vertical, horizontal, or at diagonal. A given strip sequence would progress (sweep) across the screen at an angle that was orthogonal to the orientation that had been chosen. The sweep direction was chosen at random. Location of the letter was eccentric to the center of the screen, with the location being randomly selected for each trial, as specified above. 

Nine participants provided the data for Experiment 1A. Letter contrast was varied from −1 to −20 in one-unit steps, providing progressively darker letters against the background that was a dark gray. Each letter contrast was displayed for 20 trials (20 levels × 20 trials/level = 400 total trials). The goal was to model the probability of letter recognition as a function of letter contrast for each participant and quantify how recognition probabilities varied across contrasts, participants, and other experiment- or participant-specific attributes. Bayesian generalized additive models were applied to infer these letter probability functions for each participant, including both point estimates and uncertainty quantification. The modeling provided evidence for individual differences in contrast sensitivity, in that the range of contrasts that yielded a transition from chance levels to reliable recognition often did not overlap from one individual to another. 

Experiment 1B used the same protocol, testing three of the participants for an additional two sessions looking for possible practice effects. The choice of participants was arbitrary, based on availability at a time that was convenient for experimenter and participant alike. The two additional sessions were conducted on separate test days. 

Controlling for contrast sensitivity

Varying the contrast at which the letter fragments are displayed provides a way to quantify perceptibility of stimuli (Pelli, Palomares, & Majaj, 2004; Turvey, 1973) and, as implemented here, alters the ability of participants to identify which letter has been displayed. Experiment 1 found individual differences in contrast sensitivity which seemed likely to carry forth to other experiments. To control for this factor, we required each participant in the subsequent experiments to first be tested using the Experiment 1 protocol. We then fit a probability of recognition model and determined the contrast at which the participant had an 85% probability of letter recognition. This was designated as the Lc85 index. Displaying the letter fragments at a contrast that is just below reliable (100%) recognition provides a more sensitive index of information persistence, as discussed below. In each of the following experiments, stimulus displays were based on the Lc85 of each participant as a control for individual differences in contrast sensitivity. The Lc85 index was not restricted to integer values. Fractional monitor levels were approximated by proportional sampling to average out at the desired level; for example, given a desired level of 50.2 units, for every 10 trials, eight were done at 50 units and two were done at 51 units. 

Experiment 2

Experiment 2 evaluated the effectiveness of light and dark masking of letter fragments, this being done using mask strips. Letter and mask strips were coincident (i.e., each strip covered the same zone). Figure 5 illustrates the frame-sequence conditions of Experiment 2, wherein each frame displaying letter fragments was followed by a frame that displayed a mask strip. This dual sequence was repeated at each adjacent strip location, beginning at one edge of the display screen and sweeping across to the opposite side. The key masking occurred as the sequence passed across the region where letter fragments were shown. Previous work has shown this to be an effective way to impair integration of letter fragments, thus reducing identification of the letter (Zhang et al., 2022). The degree to which this impairs letter recognition, relative to the no mask condition of Experiment 1, can be described as “strip masking.” 

Figure 5.

 

Experiment 2 displayed alternate strips of letter fragments and mask patterns, each mask strip falling across the location of the letter strip that preceded it. The contrast of mask patterns was varied from dark to light, with the extremes being illustrated in the left and right time-frame sequences.

In Experiment 2, the letter fragments were displayed at twice the Lc85 contrast that had been established for each of the participants. The two sessions were done on separate days. Pilot work had found recognition performance to be erratic when stimuli were displayed at Lc85 contrast levels, as the mask essentially served to override the threshold for letter perceptibility. However, masking effects were clearly manifested when letters were displayed at double the Lc85 level. Further, the pilot work also found that individual differences in contrast sensitivity were still present when the contrast was doubled. 

The key variable in Experiment 2 was mask contrast, which displayed the masks as light or dark departures from background intensity. Three naïve participants provided data for this experiment. The contrast level of masks was varied in small increments that ranged from being 0.2× the participant's contrast level to 2.0× the participant's contrast level. For example, if a participant had an Lc85 of 24, the mask contrast would range from 4.8 to 48 on either side of the dark/light range. A shortcoming in the experiment design was to distribute trials using ratios of monitor units instead of true luminance. However, the experiments were done at the lower end of the monitor range, where the gamma curve is relatively flat. 

Experiment 3

Experiment 3 evaluated the relative effectiveness of light and dark masks that were displayed under full-field masking conditions, the difference being the global nature of the mask that was displayed. The successive letter strips were displayed, stepping from one to the next across the monitor screen, as in earlier experiments. But, in the alternate time frames devoted to the mask condition, the full screen was filled with the mask pattern (full-field masking). Distractor letter fragments were used to create the full-field mask as they have been shown to be more effective than non-letter fragments (Jordan, 1995). Figure 6 illustrates the frame-sequence conditions of Experiment 3. 

Figure 6.

 

Experiment 3 displayed the sequence of letter fragments alternating with a mask pattern that filled the full display. Contrast of the mask pattern was varied from dark to light, as illustrated in the left and right time-frame sequences.

Three new participants were recruited and tested with the Experiment 1 protocol to derive an Lc85 contrast for each; on a subsequent day they were tested with the Experiment 3 protocol. Letter and mask contrast levels were the same as for Experiment 2. Twenty trials were provided for each of the 20 contrast levels, for a total of 400 trials. A different mask pattern was displayed for each of the 400 trials. 

Experiment 4

Experiment 4 again used mask strips to evaluate the role of spatiotemporal proximity in the ability of masks to impair recognition. Figure 7 illustrates the frame-sequence conditions of the experiment. This experiment used dark mask strips, which had effectively impaired letter recognition in Experiment 2. The time frames alternately displayed a sequence of adjacent letter strips and mask strips; however, here there were differential leads or lags for when the sequence of mask strips was launched. At one extreme, the sequence of mask frames was initiated first, which delivered five mask strips before the sequence of letter strips was started. With this head start, the mask strips encountered the letter zone and continued across it for five frames (steps) before the first set of letter fragments was displayed. This treatment condition can be described as providing a mask sequence that leads the letter-fragment sequence. 

Figure 7.

 

Experiment 4 alternated dark mask strips with letter strips, varying the spatial separation of the mask and letter strips. Here, the time frames are illustrating image content in the vicinity of the letter. The relative positioning of letter strips and mask strips is shown on each frame, and latent (unseen) images of the letter or mask patterns are shown with gray contour lines. To better illustrate the differential positions of letter and mask strips, the strips displaying letter fragments are tinted green and strips displaying mask patterns are tinted pink. The left sequence shows mask strips that were launched two time frames (steps) before starting the sequence of letter strips, providing two steps of spatial separation. The right sequence shows mask strips that are following the letter strips by two steps. In Experiment 4, the leading and lagging conditions provided up to five steps of spatial separation of mask strips relative to letter strips.

At the other extreme, the sequence of letter strips was launched, moving across five letter strips before a mask sequence was started. Here, the mask sequence can be described as lagging the letter sequence. Intermediate amounts of lead and lag were provided in Experiment 4 as specified below. 

Three new participants were recruited, each being tested with the Experiment 1 protocol to derive Lc85 indices, with another session on a subsequent day that administered the Experiment 4 treatments. We specified leading masks as negative steps and lagging masks as positive. There were 11 treatment levels: −5, −4, −3, −2, −1, 0, 1, 2, 3, 4, and 5. Each level was displayed for 38 trials, thus providing 418 total trials (11 × 38 = 418). A different mask pattern was displayed for each of the 418 trials. 

Statistical analysis

Letter recognition was modeled using Bayesian multilevel logistic regression with additive smooth effects. For each participant's i and trial j, the probability of correct letter identification, p_ij, was modeled as  

\begin{equation*}\left\{ {\begin{array}{@{}*{1}{c}@{}} {{{\rm{\pi }}_{ij}} = \frac{{{p_{ij}} - c}}{{1 - c}}}\\ {\log \left( {\frac{{{{\rm{\pi }}_{ij}}}}{{1 - {{\rm{\pi }}_{ij}}}}} \right) = z_{ij}^{\rm{^{\prime}}}\beta + {f_i}\left( {{x_{ij}}} \right)} \end{array}} \right.\end{equation*}

 

The model features several key ingredients. First, c = 1/26 adjusts for guessing; the modeled probability π_ij is between 0 and 1, and the letter identification probability p_ij is between c and 1. As a consequence, each participant can do no worse than guessing a letter uniformly at random. Second, the log-odds incorporates an adjustment for linear variables z_ij, which include the number of fragments, offset direction, and sweep direction for that trial. Finally, the model features a smooth and possibly nonlinear function, f_i(x_ij) of contrast x_ij, which admits greater flexibility than more restrictive linear specifications. 

The contrast effect is participant specific to capture each individual's letter recognition ability across different contrasts. The participant-specific smooth functions are centered at a population effect, where f_i(x) = f₀(x) + ω_i(x), which admits information-sharing among participants and provides population-level inference. Each smooth term was modeled using thin plate regression splines. If a participant is tested in multiple sessions, the addition of session-level smooth effects for each participant can also be incorporated into the model. 

Using a Bayesian approach, the unknown parameters were estimated using Markov chain Monte Carlo implemented in the brms package in R (R Foundation for Statistical Computing, Vienna, Austria). Covariates were centered as part of the model fitting process. The linear parameters β were assigned standard Gaussian priors, and the spline coefficients were modeled as conditionally Gaussian with mean 0 and standard deviations assigned a half-Student’s t-prior with 3 degrees of freedom and scale parameter 2.5. Results were robust to these choices. Crucially, the Bayesian approach provides full posterior uncertainty quantification for all unknowns. 

The Lc85 contrast level for each participant was inferred from the model by determining the contrast x for which the letter recognition probability is p_i = 85%. This quantity depends on the function f_i and therefore inherits a posterior distribution under the model. We summarized the Lc85 using the posterior mean and the 95% highest posterior density interval. Note that this quantity is computed without the linear covariates. 

For inference on the contrast effect, we adopted global Bayesian p values (GBPVs) from Meyer, Coull, Versace, Cinciripini, and Morris (2015). Informally, this strategy considers whether simultaneous credible bands for the contrast effect f_i(x) (i.e., taken jointly across all contrast levels x) exclude 0 at any point x. Intervals that exclude 0 suggest a statistically significant effect. Specifically, the GBPV is the smallest α for which the 100(1 – α)% simultaneous credible bands for f_i(x) xclude 0. Compared to classical p values, which similarly may be expressed using confidence intervals that exclude zero, GBPVs are a natural Bayesian analog. 

Results

Experiment 1

In Experiment 1A, nine participants were asked to identify letters that were displayed against a dark gray background, with the letters being darker than the background. The contrast of letters was varied across trials, and a probability of recognition function was derived for each of the nine participants based on their accuracy at each contrast level. These functions were modeled using the 1/0 data provided by each participant, where 1 indicated that the letter was correctly named and 0 indicated that it could not be identified. 

Figure 8 shows the probability of recognition function for each of these participants, with confidence intervals reflecting the degree of variability of decisions. Each individual probability of recognition function is superimposed on a plot of the group function. The effect of contrast on probability of recognition was significant across all participants (p < 0.001). Number of fragments, sweep direction, and offset direction of the letter did not have significant effects. There was variation in contrast sensitivity, with the recognition models of participants 2, 7, and 8 being significantly different from the group model (p < 0.001, p < 0.001, and p < 0.02, respectively). 

Figure 8.

 

Bayesian models of the probability of letter recognition with 95% credible bands for the group of nine participants are shown in orange, and the models for each individual are shown in blue. Contrast was varied using monitor steps, with the Weber contrast for the −5, −10, −15, and −20 levels being −0.12, −0.24, −0.35, and −0.45, respectively.

We were interested in establishing whether the differences in contrast sensitivity that were manifested in a single test session would prove to be a robust attribute of individual participants. This was evaluated in Experiment 1B, where three of the Experiment 1 participants provided data in two additional test sessions using the same display protocol. Participants were selected based on availability. The experimental protocol for Experiment 1B was identical to that of Experiment 1A, and the resulting frequency of the recognition models is shown in Figure 8. 

The low variability (thin confidence bands) that characterized the models of individual participants changed very little across the three test sessions. Contrast sensitivity of participant 5 did not differ significantly from the group for any of the three sessions (p = 0.599, p = 0.971, and p = 0.438 for sessions 1, 2, and 3, respectively). Although participant 2 differed from the group in session 1 (p < 0.001), the difference was not significant (p = 0.399 and p = 0.341 for sessions 2 and 3, respectively). Participant 7 manifested greater contrast sensitivity relative to the group across all three sessions (p < 0.001, p < 0.009, and p < 0.012, respectively). Overall, the results show substantial stability of task performance, but there were also individual differences in contrast sensitivity that might be a factor in letter recognition. 

Given that contrast sensitivity can differ from one individual to the next, it seemed prudent to control for the level of sensitivity in specifying contrast levels for the masking experiments that followed. For each subsequent experiment, participants were first tested with the Experiment 1 protocol, and probability of recognition models were derived. From each model, a contrast-sensitivity index (Lc85) was specified, which was the contrast that would be expected to produce an 85% probability of letter recognition with the same stimulus and test conditions. Experiments 2 to 4 based the contrast of letter and mask elements on the Lc85 target index. 

Experiment 2

Figure 9 shows the decline in recognition that was produced by the mask treatments. The increments shown on the abscissa are in unsigned monitor units, and the equivalent Weber contrasts for dark masks were −10 (−0.24), −20 (−0.45), −30 (−0.62), −40 (−0.72), and −50 (−0.81). For light masks, the Weber contrasts were 10 (0.18), 20 (0.58), 30 (1.09), 40 (1.66), and 50 (2.23). 

Figure 9.

 

Participants 2, 5, and 7 were retested for three sessions with the Experiment 1 protocol. The three sessions were plotted above with 95% credible bands. Characteristics of individual probability-of-recognition models were very similar across sessions, suggesting that practice did not substantially modify the ability to reliably execute the task demands. Models for participants 2 and 7 revealed contrast sensitivities that were significantly different from the group in session 1. The differences remained significant for participant 7 in sessions 2 and 3, but not for participant 2.

Figure 10.

 

In Experiment 2, each of the three participants manifested a steep decline of recognition as a function of dark mask contrast, with light masks yielding only a modest decline. Contrast increments are unsigned monitor units; Weber contrast levels are specified in the text.

Figure 10 shows that dark contrasts produced the most dramatic decline, ranging from above the Lc85 target to near chance levels of recognition where the mask contrast matched that of the letter fragments. For all three participants, the decline in accuracy using dark masks was significant (p < 0.001). The difference between the effects of light and dark contrast was also significant across all three participants (p < 0.001). Masks with light contrast produced far less impairment of recognition, with the light mask decline reaching near significance only for participant 10 (p = 0.083). We note that the dark masks were more effective even though their contrast, calculated using monitor units, was smaller than the contrast of light masks. Their effectiveness would be expected to be even stronger if the light and dark masks had been provided with equal contrast. 

Experiment 3

Three naïve participants provided data for Experiment 3. They were first tested using the Experiment 1 protocol, which provided each of them with a probability of recognition function. The Lc85 target contrast was derived, and twice that contrast level was used for display of letters across every trial of Experiment 3. 

Experiment 3 was designed to see if a full-field mask would produce greater impairment of letter recognition. The full-field mask was used to determine whether the masking is based on localized disruption of contour integration or a more generalized, global, impairment of information processing mechanisms. The same contrast range of light and dark masks from Experiment 2 were used in Experiment 3. 

Figure 11 shows the models for probability of recognition as a function of mask contrast. Letter recognition was generally lower across the full range of mask contrast for light and dark masks alike. For all three participants, neither polarity lead to significant declines in accuracy, regardless of contrast level (light: p = 0.852, p = 0.954, and p = 0.952; dark: p = 0.307, p = 0.969, and p = 0.812). Furthermore, the differences between light and dark masks in disrupting the recognition rate were not significant for any of the three participants (p = 0.220, p = 0.892, and p = 0.893). 

Figure 11.

 

The full-field mask conditions of Experiment 3 impaired letter recognition, but the dark mask patterns did not produce the large declines in recognition that were seen in Experiment 2. Contrast increments are unsigned monitor units; Weber contrast levels are specified in the text.

Experiment 4

In Experiment 2, the dark mask strips produced substantial impairment of letter recognition as a function of mask contrast, but the full-field dark masks of Experiment 3 did not produce similar effects. This suggests that spatial targeting of the mask pattern is a critical factor for disrupting the cues provided by letter fragments. To examine the role of spatial influence more thoroughly, Experiment 4 provided differential spatial displacement of the mask strips. The time frames that displayed mask strips delivered them to locations at which letter fragments were displayed, but at locations that came some number of steps before or after the sequence of letter fragments was delivered. The displacement ranged from five steps ahead to five steps after the display of letter fragments. 

Three naïve participants were tested with the Experiment 4 task, anticipating that the effectiveness of the mask would be a function of the displacement size. We expected to see minimal masking of recognition for the largest displacements (i.e., leading the letter fragment locations by five steps or lagging behind by five steps). The mask effects were expected to be greatest at zero displacement, where the frame that immediately followed the display of letter fragments displayed the mask strip at the same location. The models of the three participants who were tested with the Experiment 4 protocol are provided in Figure 12. The degree of recognition impairment was generally as predicted. Participant 16 showed a slight drop in accuracy at the extreme end of the of the displacement curve. The other two participants provided models that were expected. 

Figure 12.

 

The models for the three participants of Experiment 4 show that masking is most effective when the mask strip is displayed in close spatial proximity display of letter fragments. The probability of recognition increases as the spatial separation is increased by having the mask strips either precede or follow the letter strips (specified with negative and positive numbers, respectively).

Discussion

The first experiment found significant individual differences in contrast sensitivity among participants. This is in line with prior research that has found that individual performances varies widely as a function of luminance and eccentricity (Garcia-Perez & Peli, 1999). Individual contrast levels have been shown to be more reliable than group averages in visual experiments as the threshold of sensitivity can greatly affect recognition (Baker, 2013; Pelli & Bex, 2013). The first experiment established the individual contrast sensitivities of participants and set a baseline for within-person recognition differences. 

It is well established that primary visual cortex has neurons that are designed to register contours (Hubel & Wiesel, 1959; Kara & Reid, 2003; Snodderly & Gur, 1995), with a given neuron being especially responsive to a contour that lies at the orientation of its elongated receptive field. In our case, the contours were the fragments that made up each target letter. What is less well appreciated is the fact that the receptive fields of V1 neurons are relatively short, with the average length being about 1° of visual angle and with the longest being about 2° (Sceniak, Hawken, & Shapley, 2001). Letter height at the viewing distance for the present experiments was 4.2° of visual angle, ensuring that contours would have to be registered by many cortical neurons and integrated to achieve recognition of the full shape. 

Classic work in perception has shown that a full image could be recognized from incomplete parts (DiLollo & Wilson, 1978). The integration of fragments could be seen at very low contrast levels—a few long contours could be seen even for the faintest images. Cooperative interactions among orientation-selective neurons in V1 may provide for integration of the letter fragments. Many laboratories have demonstrated that neurons can synchronize with precision in the millisecond range and do so especially when the stimulus elements are collinear (Aertsen & Arndt, 1993; Engel et al., 1992; Singer & Gray, 1995). Precise synchronization of spatially separate neurons has been demonstrated in striate and extrastriate cortex of primates (Kreiter & Singer, 1992; Ts'o & Gilbert, 1988). 

Perhaps entrained oscillations provide for long-range linkage as well as synchronization of activity over extended time intervals (König et al., 1995). Castelo-Branco and associates (1998) reported that flashed bars or gratings triggered high-frequency (60–120 Hz) gamma oscillations in the retina and lateral geniculate nucleus that lasted for about a second. Fries et al. (2001) recorded local field potentials from a population of orientation-selective cells in V1 and found that spatially contiguous contours produced gamma waves in this population that were implicated in joint processing of rapidly shown visual stimuli. Within our experiment, we wanted to explore the role of integration over many fragments in recognition of letters and whether integration of every letter fragment was necessary for recognition. Our use of multiple letter fragments deepens this understanding of the limits of integration of a single target. 

Experiment 2 used strips of incomplete letter fragments as a mask for the target letters to test the effects of mask contrast on recognition across dark and light scales. Masking has been very well studied as a phenomenon that disrupts vision, typically involving a briefly flashed target that is quickly followed by a mask (Sperling, 1963; Turvey, 1973). Many types of masking studies have been performed to examine the effects of contrast polarity, stimulus onset asynchrony, and spatial contiguity on recognition (Enns & DiLollo, 2000; Lamme et al., 2002). More recent work has used masking paradigms in combination with physiological techniques, such as electroencephalography (EEG), to look at the neural correlates of perceptual states (Fahrenfort, Scholte, & Lamme, 2007; Koivisto & Revonsuo, 2010). Work with EEG has shown that there are distinct event-related potentials when objects obstructed by a mask appear in visual awareness (Fahrenfort et al. 2007; Koivisto & Grassini, 2016; Wilenius-Emet, Revonsuo, & Ojanen, 2004). Other studies suggest that there are later visual signals that are affected by backward masking, in which a mask is shown after a target stimulus (Koivisto & Grassini, 2016). 

Experiment 2 shares similarities with metacontrast masking studies, in which a target stimulus and mask did not overlap but the mask was still effective (e.g., Enns & DiLollo, 2000; Kolers & Rosner, 1960). The effectiveness of masking stimuli in smaller fragments, rather than a full-field mask, has been shown in many metacontrast studies, in which a small non-overlapping mask has been flashed after the target (e.g., Breitmeyer, Hoar, Randall, & Conte, 1984; Enns & DiLollo, 2000; Ogmen et al., 2003). Unlike most previous studies, however, to see the full figure, our participants had to not only separate the target stimulus from a mask but also repeat that procedure over many steps. Providing the letter as a fragment sequence ensured that integration of persistence shape cues was required for successful recognition of the letter. Mask strips having sufficiently high contrast may have acted to weaken or override the residual trace of the prior letter fragments and therefore were particularly effective in disrupting recognition. We theorize that the strip masks are acting to dampen the persistence, with strips of higher contrast providing more effective suppression of persistence. 

Early work in metacontrast masking by Michaels and Turvey (1979) showed differential effects of mask contrast on recognition using dark masks. Their results showed that masks with a greater contrast were overall more effective, regardless of letter contrast. Pelli and associates (2004) showed that reducing the contrast ratio of letters to masks leads to reduced recognition of target letters when non-target letters provide the distractors. In that study, letters were shown on a screen and flanked by two non-target letter contrasts that had varying luminance levels, up to 0.85× the target. The experiment used light letters, but the monotonic decline of recognition that was produced as same-polarity mask contrast increased suggests that the match in polarity and the contrast ratio between the target stimuli and mask both play a key role in effectiveness of a distractor toward recognition of a target stimulus. 

There is reason to believe that integration of fragments having the same contrast polarity occurs in V1. Many orientation-selective neurons in the V1 are specific to the contrast polarity—some respond to light bars and some respond to dark bars (Celebrini et al., 1993; Hubel & Wiesel, 1959; Sceniak et al., 1999). In registering a given shape, one would expect neighboring receptive fields to preferentially link self-similar fragments, thus leading to greater effect of masking in the dark condition, in which the mask fragments were linked into an overall nonsense image. Following this, the lack of effect of the light mask strips, regardless of contrast level, indicates a substantial degree of independence of the mechanism by which dark and light fragments are linked. 

Prior work by Becker and Anstis (2004) showed that the effect of masking was dependent on color polarity in masking shapes. Dark and light target circles were used, with a surrounding mask flashed after the target stimulus that either matched or did not match the polarity of the target. In their study, researchers found that masking was only effective when the mask matched the polarity of the target shape. This has been attributed to the differential activation in ON and OFF pathways within the V1, depending on an increase or decrease of luminance (Arnold & Anstis, 1993). It has been suggested that this differential activation causes separate, distinguishable signals when the mask does not match the polarity of the target (Becker & Anstis, 2004). In the present study, this polarity effect held consistently over multiple fragments. Our results suggest that the global summary derived from these signals differentially activates the ON/OFF channels of the V1 which stay separate throughout the entire recognition process, allowing for recognition of the letter to be integrated completely only in the light mask condition. 

Experiment 3 found that both dark and light full-field masks had very weak effects on recognition. What is most surprising is the lack of effect within the dark mask condition compared with the large effect seen in Experiment 2. This indicates, perhaps counterintuitively, that smaller masks strips are more effective at disrupting recognition than full-field masks. The weak effect of the full-field mask may be attributed to the role of transient versus sustained channels in the V1, rather than the ON/OFF cells that are responsible for the effects seen in Experiment 2. Within the time frame given in our experiment, it is likely that this initial encoding process is happening due to the many transient and sustained channels that are activated by either steady or transient signals (Tolhurst, 1975). 

It is important to note that, in our experiment, the mask pattern remained the same for a given trial. Repeating the image pattern in each full-field mask frame may have created a steady background through activation of sustained channels, such that the letter fragments could be recognized due to activation of the transient channels. It was proposed by Breitmeyer and Ganz (1976) that that sustained channels are involved in the processing of structural or figural information, whereas transient channels are involved in signaling the spatial location or change in spatial location. Others have noted that this dichotomy is temporally separate, with the transient channels reacting more quickly to stimuli than the sustained channels (Mitov, Vassilev, & Manahilov, 1981). Despite the many fragments being integrated, it seems that signals from the transient channels are easily distinguishable, even over hundreds of milliseconds. 

Our results suggest that V1 can process the different signals in a parallel format, separating the temporal, spatial, and luminance factors. There are many transient and sustained channels within V1, as well as ON/OFF channels, that would allow for this separation (Tolhurst, 1975). The results support the idea that V1 can parallel process the different signals coming in and separate the noise from the target. This model would be consistent with the differences in mask type and their results analyzed in prior review of masking (Breitmeyer et al., 2006; Enns & DiLollo, 2000). However, the weak effects of the full-field mask in the third experiment go against much of the literature that has used full-field masks to great effect (Lamme et al. 2002; Turvey, 1973). Some models of masking focus on whether masking affects the feedforward processes required for integration or feedback processes that disrupt the decoding of a comprehensive signal from the target (Goodhew, 2017; Koivisto & Revonsuo, 2010). Experiment 3 demonstrated that information being integrated across letter fragments is sufficient for participants to retrieve those letters from memory. 

Full-field masking did produce a lower overall level of recognition, but there was no differential effect of contrast, which suggests that the decline may be attributed to a simple increase in timing between target letter fragments or crowding effects. Crowding, a visual phenomenon by which a target is masked by the presence of other flanking distractors, has been shown to be effective when flashed in close proximity to target stimuli (Averbach & Coriell, 1961; Strasburger, Harvey, & Rentschler, 1991). With the many different fragments seen onscreen during the mask frames, it is likely that there are effects of visual phenomena unrelated to disruption of integration. 

The fourth experiment found that masking became less effective with an increase in lead and lag distance from the target letter strip. The U-shaped curve result is very similar to that found in experiments in which stimulus onset timing between the mask and the target was varied (Bachmann, 1994). Use of lagging and leading masks was similar to that of paracontrast and metacontrast masking, in which the mask either followed or preceded the target in time (Breitmeyer et al., 2006; Eriksen, 1966). Both methods have been shown to be effective (Eriksen, 1966), and we did not find any significant difference in performance between the leading and lagging strip trials. This lack of difference could be due to the number of strips within our experiment rather than a single target stimulus. 

Based on the results of our study, it seems the initial integration of fragments is happening locally with adjacent letter fragments in which both spatial and temporal information play an important role in ease of integration and effectiveness of distractors. The results of our study suggest that spatial information of the target, such as distance from the distractor, plays an important role in the effectiveness of strip masking. Our results are consistent with crowding research that has shown that distance from a target affects the recognition of a target stimulus (Bouma, 1970; Pelli, 2008). Other work with crowding distance using letters has found similar results in which distractors were more effective in central crowding versus peripheral crowding (Leat et al., 1999). Although there is discussion about a critical state at which crowding is effective, our work shows that the effect is more aligned upon a continuum. This is in line with the theory that the positional codes of the mask and the target are taken into account and can be differentiated based on distance from each other (Kooi et al., 1994). 

Taken together with the results of the prior experiments, immediacy and spatial co-location are key factors in integration of shape cues. By using strips, we provided masking in which each mask strip acted on the letter fragments presented close to their proximity. The results suggest that impairment of letter recognition from a mask-strip sequence is produced by the aggregation of localized actions. 

Acknowledgments

Funding was provided by the Quest for Truth Foundation and the Neuropsychology Foundation. Tailai Shen assisted in coordination of test administration and data processing. 

Commercial relationships: none. 

Corresponding author: Sherry Zhang. 

Email: [e-mail]. 

Address: Department of Psychology, University of Southern California, Los Angeles, CA 90089, USA.