Interference in character processing reflects common perceptual expertise across writing systems

Abstract

Perceptual expertise, even within the visual domain, can take many forms, depending on the goals of the practiced task and the visual information available to support performance. Given the same goals, expertise for different categories can recruit common perceptual resources, which could lead to interference during concurrent processing. We measured whether irrelevant characters of one writing system produce interference during visual search for characters of another writing system, as a function of expertise. Chinese–English bilinguals and English readers searched for target Roman letters among other distractors in a rapid serial visual presentation (RSVP) sequence. Chinese character distractors interfered with Roman letter search more than pseudoletter distractors, only for bilingual readers, suggesting a common perceptual bottleneck for Roman and Chinese processing in experts with both domains. We ruled out an explanation at the level of phonetic codes, by showing that concurrent verbal rehearsal has no effect on the magnitude of such interference. These findings converge with results showing competition between faces and cars in car experts to suggest that different domains of expertise that overlap in their cortical representations also possess a common perceptual bottleneck.

Introduction

The notion of perceptual expertise is increasingly helpful for understanding apparent qualitative differences in processing different object categories and for explaining specialization in the visual system for these categories (Bukach, Gauthier, & Tarr, 2006). The majority of research to date addresses face-like expertise, characterized by superior performance individuating objects within a visually homogeneous class. Experts with dogs, birds, cars, fingerprints, novel computer-generated objects, etc., demonstrate perceptual and neural markers once considered to be unique hallmarks of face perception (Busey & Vanderkolk, 2005; Gauthier, Skudlarski, Gore, & Anderson, 2000; Gauthier & Tarr, 2002; Tanaka, Curran, & Sheinberg, 2005; Tanaka & Taylor, 1991). Perceptual expertise, however, does not come in one type only. Differential training with the same objects can produce different sets of skills depending on the practiced task (Scott, Tanaka, Sheinberg, & Curran, 2006; Song, Hu, Li, Li, & Liu, 2010; Wong, Palmeri, & Gauthier, 2009; Wong, Palmeri, Rogers, Gore, & Gauthier, 2009). Moreover, many perceptual skills (e.g., radiograph viewing, letter and character perception, music reading, map reading, etc.) may not be readily regarded as examples of face-like expertise, yet can offer critical insight into the nature of perceptual expertise. It is therefore important to understand how expertise with different object categories can be grouped together or distinguished from each other. 

Recent work suggests a common perceptual bottleneck shared by “similar” kinds of expertise for two domains where objects are visually distinct (e.g., cars and faces): neural responses to these objects interfere with each other (Gauthier, Curran, Curby, & Collins, 2003; Rossion, Collins, Goffaux, & Curran, 2007; Rossion, Kung, & Tarr, 2004) and behavioral performance trade-off in perceptual tasks (McGugin, McKeeff, Tong, & Gauthier, in press; McKeeff, McGugin, Tong, & Gauthier, 2010). However, different visual skills can be similar on some dimensions (e.g., recognizing Roman letters and Chinese characters both require distinguishing among shapes made out of stroke patterns that differ in their structural descriptions) and dissimilar on other dimensions (e.g., the number of characters is much larger for Chinese, and the mapping on phonology obeys very different rules). How can we decide if such visual skills are similar despite some differences, or in other words, how can we unravel the dimensions that are relevant to understanding the relationships between various visual skills? Patterns of selectivity and overlap in the brain may be one way to segregate kinds of expertise, but in practice, any two domains will overlap to some extent while also recruiting separate neural territory. Another way to assess the similarity between two visual domains is to ask whether they rely on common functional resources (Kinsbourne & Hicks, 1978; McGugin et al., in press; McKeeff et al., 2010). Here, we use this approach to assess whether expert recognition of Roman letters and Chinese characters tap onto a common perceptual bottleneck. 

Expertise with text represents a specific type of perceptual expertise distinct from face perception, with its unique set of perceptual and neural markers (James, Wong, & Jobard, 2009). Whereas face perception requires subordinate-level individuation of objects that share a common part structure, letter perception requires coarser, basic-level categorization whereby small metric differences between different examples of the same letters can be ignored (Wong & Gauthier, 2007; Zhang & Cottrell, 2004). Therefore, the basic-level advantage typically found in object recognition, whereby we are faster to categorize objects as cars, dogs, and trees than as a Mustang, Labrador, or oak, is larger for letters in expert readers than in non-readers (Wong & Gauthier, 2007). This is in contrast with the reduction of the basic-level advantage found for face-like expertise (Tanaka, 2001). Expertise with text also allows one to utilize the regularity of fonts within a text to facilitate character perception (Gauthier, Wong, Hayward, & Cheung, 2006), such that one would recognize a string of letters in the same font faster than in different fonts. Letters and faces engage different brain regions in the ventral visual stream, with the former recruiting mainly left, lateral occipito-temporal regions, and the latter recruiting the middle fusiform gyrus mainly in the right temporal lobe (Hasson, Levy, Behrmann, Hendler, & Malach, 2002; Wong, Jobard, James, James, & Gauthier, 2009). In fact, the processing of letters can be dissociated from that of words and even non-word strings (James, James, Jobard, Wong, & Gauthier, 2005). 

While expertise for letters has proven to be distinct from other visual skills in several respects, relatively less is known about its generalizability, i.e., whether expertise processing individual characters from different writing systems relies on the same perceptual resources. Some evidence suggests a common system. In expert readers, increases in basic-level advantage and the facilitation experienced when characters are shown in the same font have been found not only with Roman letters but also with logographic Chinese characters (Gauthier et al., 2006; Wong & Gauthier, 2007). Functional magnetic resonance imaging (fMRI) studies reveal visual regions that are selectively activated by characters (or in some studies by words) across different writing systems like English, Hebrew, Japanese, Chinese, etc. (Baker et al., 2007; Bolger, Perfetti, & Schneider, 2005; Wong, Jobard et al., 2009). Event-related potential (ERP) studies have also identified an N170 component selective for familiar characters across writing systems (Maurer, Zevin, & McCandliss, 2008; Wong, Gauthier, Woroch, Debuse, & Curran, 2005). These findings suggest a perceptual stage on which the drastically different linguistic properties associated with these writing systems have little influence. 

Here we test whether common expert perceptual mechanisms are recruited for processing two types of characters (Roman letters and Chinese characters). We examined interference in bilinguals performing a task where one class of characters is relevant and the other class is present as irrelevant distractors whose interfering effect can be assessed. If different writing systems share common perceptual resources, then processing of characters from one writing system should compete with that of characters from another writing system. Since Roman letters and Chinese characters belong to highly different writing systems (alphabetic and logographic, respectively) and differ on many dimensions linguistically (in terms of orthography, phonology, etc.), any observed interference would more likely arise from visual processes common for these characters, rather than as a result of similarities across particular linguistic levels of processing. 

Certainly, different types of characters share some common low-level features irrespective of one's level of expertise and thereby should recruit similar early visual processes. However, here we are interested in whether expertise with different characters leads to a larger overlap of processing: when concurrently processing characters from two unique writing systems, is interference greater for individuals with expertise in both systems relative to individuals with expertise in only one? This expertise-associated interference would be difficult to explain based solely upon the similarity in low-level stimulus properties. To probe the effect of expertise, we recruited two groups of participants: English readers, who were experts with Roman letters but novices with Chinese characters, and Chinese–English bilinguals who started to learn written English and Chinese at the same time in school and were highly proficient with both writing systems. If there is a common expert perceptual mechanism generalizable to different writing systems, then processing of Chinese characters would interfere with processing of Roman letters for bilinguals but not for English readers. 

Experiment 1

Participants searched for Roman letter targets in a rapid serial visual presentation (RSVP) sequence, with either Chinese characters or pseudoletters added as distractors. This procedure has been used to show interference between faces and objects with which one has developed face-like expertise (McKeeff et al., 2010). It allows the target and distractors to appear at the same location and thus offers a potentially more sensitive measure of interference than tasks like spatial visual search and Eriksen's flanker task (Eriksen & Eriksen, 1974; Treisman & Gelade, 1980), in which the target and distractors may be easily separated by perceptual grouping based on low-level feature differences among different types of characters (Humphreys & Müller, 2000; Treisman, 1982). If different writing systems share common expert perceptual processing, then bilinguals proficient in both Roman and Chinese writing systems should show worse performance in Roman letter search among Chinese distractors than among novel pseudoletter distractors. To English readers, however, both Chinese characters and pseudoletters are novel, and therefore, Roman search performance should be similar with either type of distractors. 

Critically, it is possible that one could observe interference of Chinese characters on Roman letter search not because of common processing for the two types of characters but merely due to the fact that any familiar distractors draw attention away the targets thereby impeding performance (Awh et al., 2004). To test this alternative attention-based possibility, participants performed search for pseudoletter targets in some of the trials with either Roman letters or Chinese characters as distractors. If any familiar distractors can attract attention and thus impede search, then for English readers the familiar Roman distractors should lead to worse pseudoletter search performance compared with novel Chinese distractors. 

Methods

Participants

Twenty-four bilinguals (11 males, mean age = 20.58) were recruited at the Chinese University of Hong Kong. They all learned to read and write Chinese and English starting from approximately age 3 in school. After over 15 years of training, they were proficient in both reading and writing English, with most of their textbooks and assignments written in English. Previous work has shown equivalent levels of expertise with Roman letters—in terms of both behavioral recognition performance and neural activity level—for bilinguals (as classified here) as for English readers (Wong & Gauthier, 2007; Wong, Jobard et al., 2009). Twenty-eight English readers (13 males, mean age = 24.71) with no prior experience of learning Chinese were recruited at Vanderbilt University. All individuals had normal or corrected vision and were given monetary compensation or course credit for their participation. 

Material

Eighteen upper case Roman letters (except A, E, I, J, O, T, X, and Z), 18 Chinese characters, and 18 pseudoletters were used (Figure 1A). The Chinese characters were all valid words with meanings and pronunciations. Pseudoletters were generated by recombining strokes of Roman letters, based on the principle that they contain similar number of strokes, thereby sharing similar visual complexity. A previous ERP study (Wong et al., 2005) used half of the stimuli in the current study and showed similar visual complexity among the three types of characters: whereas English readers showed an N170 component of similar amplitudes to Chinese characters and pseudoletters, bilinguals showed similar N170 amplitudes to Roman letters and Chinese characters. Each character spanned 1.9° of visual angle with a 60-cm viewing distance. Stimuli were presented on a 19-inch CRT monitor (refresh rate = 100 Hz) with a Mac Mini using MATLAB (MathWorks, Natick, MA) and Psychophysics Toolbox (Brainard, 1997). 

Figure 1

 

(A) The Roman letters, Chinese characters, and pseudoletters used in Experiment 1 and (B) a sample trial sequence in the R/RC condition.

Procedure

Participants searched for targets in an RSVP sequence (Figure 1B). There were four conditions: Roman search among Roman and Chinese distractors (R/RC), Roman search among Roman and Pseudoletter distractors (R/RP), Pseudoletter search among Pseudoletter and Roman distractors (P/PR), and Pseudoletter search among Pseudoletter and Chinese distractors (P/PC). There were 30 RSVP sequences in each block and three blocks for each of the four search conditions. Block order was counterbalanced across participants. 

On each trial, two targets were randomly selected and presented side by side for participants to view for as long as necessary. Upon pressing the space bar, a blank screen appeared for 1 s, followed by an RSVP sequence of 20 images alternating between two categories. Only one of the two targets appeared once in the RSVP sequence and the target never appeared as the first or last image. After the RSVP sequence, the two targets reappeared and participants indicated which of them they spotted in the sequence by pressing the “F” or “J” key with their left and right index fingers, respectively. A staircase procedure manipulating presentation rate for each image was used to measure the presentation speed (images per second) at which 82% accuracy was achieved in each block (Watson & Pelli, 1983). Average threshold speed was computed across blocks for each search condition. A higher threshold speed indicates better search performance. 

Results and discussion

When searching for Roman targets (Figure 2A), bilinguals performed worse when the RSVP stream was embedded with Chinese distractors (R/RC) compared with Pseudoletter distractors (R/RP), as indicated by a lower threshold speed for the R/RC condition. English readers, however, performed similarly with the two types of distractors. A Group (bilinguals, English readers) × Search Condition (R/RC, R/RP) analysis of variance (ANOVA) showed a main effect of Search Condition [F(1, 50) = 8.502, p < 0.01, η _p ² = 0.145] and an interaction between Group and Search Condition [F(1, 50) = 4.014, p = 0.05, η _p ² = 0.074]. The main effect of Group was not significant [F(1, 50) = 1.921, p = 0.17, η _p ² = 0.037]. Scheffé's tests (p < 0.05) showed that the threshold speed was lower in R/RC than R/RP condition for bilinguals but not English readers. In addition, the bilinguals had a lower threshold speed than the English readers only in the R/RC but not the R/RP condition. The similar level of performance in the R/RP condition between the two groups is not surprising, given that the bilinguals learned to recognize and write Roman letters at about the same age as they learned Chinese characters. 

Figure 2

 

Search performance for (A) Roman letters and (B) Pseudoletters in Experiment 1. Error bars represent 95% confidence interval for the search condition factor.

A possible account for our findings is that the bilinguals paid more attention to the familiar Chinese distractors, which subsequently impeded their search for Roman letters. Following this logic, one may expect that, for English readers, Pseudoletter search should be impaired by Roman distractors more than Chinese distractors, since Roman letters were more familiar to English readers. However, no such difference was observed; in fact, if anything, performance was numerically better with Roman relative to Chinese distractors (Figure 2B). A Group (bilinguals, English readers) × Search Condition (P/PC, P/PR) ANOVA showed no significant main effect or interaction (ps > 0.19). Therefore, the additional interference of Chinese characters (as compared with Pseudoletters) on Roman letter search in bilinguals cannot be explained simply by the familiarity of the distractors themselves. 

It should be noted that performance was better for Roman search than Pseudoletter search (threshold speed: 8.65 vs. 6.99, F(1, 50) = 45.08, p < 0.0001, η _p ² = 0.474; no interaction with Group, F < 1). The perceptual load during Pseudoletter search may have been so high that it exhausted the attentional resources of observers, leaving no room for any interference from Roman distractors on Pseudoletter search to be shown for the English readers (Lavie, 1995). We therefore filtered out about half of the participants (14 for each group) who had superior performance in Roman compared with Pseudoletter search to the largest extent. For the remaining participants, performance was no longer different between Roman and Pseudoletter searches (threshold speed: 8.35 vs. 8.08, F(1, 22) = 1.96, p > 0.17, η _p ² = 0.081; no interaction with Group, F < 1). Yet the same result pattern was obtained: For Roman search, a Group × Search Condition (R/RC, R/RP) interaction was found [F(1, 22) = 8.42, p < 0.01, η _p ² = 0.276], with the difference between R/RC and R/RP condition only significant for bilinguals in Scheffé's tests (p < 0.05); for Pseudoletter search, a Group × Search Condition (P/PC, P/PR) ANOVA again showed no main effect or interaction [Fs < 1]. Therefore, the likely explanation of the lack of impairment of Pseudoletter search by Roman distractors for English readers is the little overlap in their processing, instead of the higher perceptual load for Pseudoletter search. 

Experiment 2

Experiment 1 showed that search performance was impaired only when both targets and distractors engage expert processing mechanisms. The search task involved online visual comparison across targets and distractor items in a rapid visual sequence, with no requirement as to the use of linguistic knowledge regarding the stimuli. Yet one could suggest that the bilinguals demonstrated Roman–Chinese interference because they possessed verbal labels for characters of both writing systems. Interference between Roman letters and Chinese characters could hence have occurred at the level of phonetic codes rather than visual codes (Posner, 1978; Posner, Boies, Eichelman, & Taylor, 1969). To address this issue, in Experiment 2, bilinguals performed Roman target search among Chinese or pseudoletter distractors, with half of the blocks requiring concurrent verbal rehearsal of four digits. If the Roman–Chinese interference found in Experiment 1 occurred partly at the level of phonetic codes, then having a verbal load would reduce the interference by disrupting phonetic code representations. If the Roman–Chinese interference found in Experiment 1 was mainly of a perceptual nature, then a verbal load would only reduce the overall performance level but not the interference. 

Methods

Participants

Thirty-three Chinese–English bilinguals (17 males, mean age = 20.52) who learned to read and write Chinese and English starting from around age 3 in school were recruited at the Chinese University of Hong Kong. All had normal or corrected vision and were given monetary compensation or course credit for their participation. 

Material

The same characters and computers used in Experiment 1 were used here. Digits 1 to 9 were used for verbal rehearsal. 

Procedure

Participants again searched for targets in RSVP sequences and threshold speeds were measured. The targets were always Roman letters and there were two search conditions: Roman search among Roman and Chinese distractors (R/RC) and Roman search among Roman and Pseudoletter distractors (R/RP). “F” and “J” keys were used for same and different responses, respectively. There were eight 30-trial blocks for each search condition, and half of the blocks required concurrent verbal rehearsal of digits. Block order was counterbalanced across participants. 

For blocks with verbal rehearsal, on the first trial, four digits were shown on the screen for 4 s, which participants had to rehearse in English throughout the trial. Then, the study screen, RSVP sequence, and response screen were presented in the same manner as in Experiment 1. After participants made a response to the targets, four digits were shown and participants pressed the “V” or “N” key with their left and right fingers, respectively, to indicate if the four digits were the same as the ones being rehearsed. No digits were shown at the beginning of the second trial, as the same four digits were rehearsed. A new set of four digits was used after every 10 trials. For blocks without verbal rehearsal, each trial began with the words “No need to rehearse” for 4 s and ended with a screen asking the participant to press either the “V” or “N” key to continue. In this way, the presentation sequence and responses required were kept as similar as possible for the blocks with and without verbal rehearsal. 

Results and discussion

Overall search performance was worse with relative to without a verbal load as expected. However, the verbal load caused no difference in the magnitude of interference of Chinese distractors on Roman letter search as compared with Pseudoletter distractors (Figure 3). A Verbal Load (load, no load) × Search Condition (R/RC, R/RP) ANOVA showed a main effect of Search Condition [F(1, 32) = 5.288, p < 0.05, η _p ² = 0.142] and a marginally significant effect of Load [F(1, 32) = 3.944, p = 0.056, η _p ² = 0.110] but no interaction [F < 1; η _p ² < 0.01]. Scheffé's tests (p < 0.05) showed that the threshold speed was lower in R/RC than R/RP condition whether there was a load or not. Such a lack of interaction was unlikely to be a result of the inefficiency of the verbal load or lack of power, since the load did lower the overall performance level. It should also be noted that performance of the bilinguals was better than those in Experiment 1 [F(1, 55) = 16.644, p < 0.0001, η _p ² = 0.110]. This baseline difference highlights the importance of comparing different search conditions within subjects, as adopted in our design. 

Figure 3

 

Search performance of bilinguals in Experiment 2. Error bars represent 95% confidence interval for the search condition factor.

General discussion

Our results suggest a common bottleneck engaged by expert processing of characters from different writing systems, one that is relatively distinct from resources used by novices when processing the same shapes. Experiment 1 showed interference of Chinese characters on Roman letter search only for bilinguals familiar with both writing systems, as indicated by the worse search performance with Chinese distractors compared with pseudoletter distractors. Such interference was not a result of Roman letters being more similar to Chinese characters than to pseudoletters, because, for English readers, search performance was similar with the two types of distractors. Nor can the interference be explained by the familiar Chinese distractors capturing bilinguals' attention, since for English readers the familiar Roman distractors did not interfere with pseudoletter search more than the unfamiliar Chinese distractors. 

The interference found between different writing systems likely reflects a processing bottleneck at a perceptual level. First, the visual search task involved merely perceptual matching and required no linguistic knowledge such as pronunciation or meaning. In fact, whereas the Chinese character stimuli represent meaningful words, the Roman letters do not carry meaning and thus interference cannot occur at the semantic level. Moreover, as shown in Experiment 2, it is unlikely that the interference simply occurred at the level of phonetic codes, as Chinese characters interfered with Roman letter search to similar extent with or without verbal suppression, even when this suppression was sufficient to impact overall search performance. Therefore, the interference found in the current study is best explained by the processing of different writing systems tapping into common expert perceptual resources. 

A perceptual expertise framework can help us understand behavioral and neural category-selective effects resulting from the association of a category's visual features and a task that has been repeatedly associated with that category (Gauthier, 2000). Within this framework, we can try to understand the computations performed by a category-selective neural network by understanding the nature of expertise in the relevant domain. Because there is in principle an infinite number of visual tasks and domains we could acquire expertise with, a taxonomy for various kinds of expertise would facilitate such inferences. For example, in the present case, knowing that there is a common bottleneck for expert perception of Roman letters and Chinese characters allows us to infer that common aspects of expertise in these domains constrain the organization of the visual system. This is consistent with neuroimaging work showing overlapping selectivity for these two domains in left occipito-temporal cortex (Wong, Jobard et al., 2009). Because selectivity for letter expertise can be distinguished from that for faces (Hasson et al., 2002; Wong, Jobard et al., 2009) and also for musical notation (Wong & Gauthier, 2010), the methods used here can be used to test the prediction that such differences in brain selectivity would result in minimal interference between these domains of expertise. 

Finally, it is important to acknowledge that non-perceptual factors may also determine the extent to which different domains of expertise share resources. For example, in the case of characters in different writing systems, the need to provide inputs to post-perceptual stages of processing necessary in reading likely constrains which neural substrates are engaged (Carr & Pollatsek, 1985; James et al., 2009). Another factor may include sensorimotor interactions, such as the role of writing in letter perception (James & Atwood, 2009; James & Gauthier, 2006) or musical performance in the perception of musical notation (Wong & Gauthier, 2010). While theories provide reasons why expertise in different domains may rely on common resources, measuring perceptual interference between domains allows one to test the predictions of such theories (Kinsbourne & Hicks, 1978; McGugin et al., in press; McKeeff et al., 2010). 

Acknowledgments

This research was supported by grants from the Chinese University of Hong Kong (Direct Grant 2020939) and the Research Grants Council of Hong Kong (General Research Fund 452209) to Alan Wong and from the Temporal Dynamics of Learning Center (SBE-0542013), the James S. McDonnell Foundation, and the National Eye Institute to Isabel Gauthier (2 R01 EY013441-06A2 and P30-EY008126). 

Commercial relationships: none. 

Corresponding author: Alan C.-N. Wong. 

Email: [email protected]. 

Address: Department of Psychology, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong.