The effects of shape crowding on grasping

Abstract

Crowding refers to the deleterious effect of nearby objects on the identification of a target in the peripheral visual field. A recent study (Chen, Sperandio, & Goodale, 2015) showed that when a three-dimensional (3D) disk was crowded by disks of different sizes, participants could scale their grip aperture to the size of the target, even when they could not perceive its size. It is still unclear, however, whether or not grasping can also escape to some degree the crowding of other object features, such as shape. To test this, we presented 3D rectangular blocks in isolation or crowded by other blocks in the periphery. The target and flanking blocks had the same surface area but different dimensions. Participants were required either to grasp the target block across its width or to estimate its width. We found that, consistent with what we observed earlier with size, participants can also scale their grasp to the width of the target block even when they could not perceive its width. To further explore whether or not the effect of crowding on grasping depends on how proficient people are with their right hand, we had right-handed participants perform the same test but with their left hand. We found that left-hand grasping did not escape the crowding effect on shape perception at all. Taken together, our results suggest that people can also use invisible shape information to guide actions and that this ability depends on the proficiency of the action.

Introduction

It is more difficult to identify features of an object when it is surrounded by other objects than when it is presented in isolation. This phenomenon is called crowding and is more evident in the periphery of the visual field (Levi, 2008). Although people have studied crowding for several decades, it is still unclear how crowding occurs. One set of theories suggests that crowding arises because the features of the different objects presented in the periphery are integrated or pooled together (Parkes, Lund, Angelucci, Solomon, & Morgan, 2001; Pelli, 2008). The pooling of features could be due to the large receptive fields of neurons representing the periphery or to their long horizontal interconnections. In either case, these theories posit that crowding occurs at the early stages of visual processing. Another set of theories suggests that crowding is due to the fact that the resolution of attention is not sufficient to separate individual features in densely distributed arrays (He, Cavanagh, & Intriligator, 1996). Using a combination of electroencephalography and functional magnetic resonance imaging, Chen et al. (2015) found that the crowding observed with gratings is due to suppressive cortical interactions between V1 neurons that code the individual flankers and target. Moreover, the investigators found that such suppressive interaction occurs early and is modulated by attention, suggesting that attention-dependent V1 suppression contributes to crowding at an early stage of visual processing. Recently, a number of new studies (Malania, Herzog, & Westheimer, 2007; Manassi, Sayim, & Herzog, 2012; Saarela, Sayim, Westheimer, & Herzog, 2009) suggest that crowding depends on the grouping of the flankers and the target. Contrary to the common observation that the effect of crowding increases with the number of flankers, these studies found that adding flankers can sometimes reduce the effect of crowding when the flankers are grouped in a particular way. This observation cannot be explained either by the pooling model or by the attention model and provides a new perspective on the crowding effect. 

Although how crowding affects the identification of features has been studied extensively, few studies have investigated how crowding affects action such as grasping (Bulakowski, Post, & Whitney, 2009; Pardhan, Gonzalez-Alvarez, Subramanian, & Chung, 2012). When people use two fingers to grasp an object, the grip aperture first increases and then decreases as the fingers close in on the object. The peak grip aperture (PGA), which occurs well before the fingers make contact, is typically scaled to the size of the goal object and is believed to be programmed before the movement of the fingers begins (Jeannerod, 1986). This phenomenon is called grip scaling. Recently, it has been shown that people can scale their grip aperture to the size of the 3D disk even when the size of the target disk is made invisible (Chen et al., 2015). This suggests that people can use invisible information about size to guide their actions. It is still unclear, however, whether or not this is also the case for other kinds of invisible information, such as crowded shape. 

Shape is another important visual cue, in addition to size, that people employ to guide actions in their everyday life. When you reach out to grasp a block, you have to consider not only its size (big or small) but also its shape (rectangular or square). Moreover, in the visual hierarchy, size and shape are represented in different areas. Whereas size is represented as early as in V1 (Fang, Boyaci, Kersten, & Murray, 2008; Murray, Boyaci, & Kersten, 2006; Sperandio, Chouinard, & Goodale, 2012), shape is believed to be represented mainly in higher-level visual areas such as V4 (Pasupathy & Connor, 2001) and lateral occipital cortex (Kourtzi & Kanwisher, 2001) of humans or the inferior temporal cortex of monkeys (Logothetis, Pauls, & Poggio, 1995). It is possible that the earlier encoding of size allows it to escape the effect of crowding. Because shape information is extracted much later, it is possible that the control of grasping will be as sensitive to crowding as perceptual report. Nevertheless, work with neurological patients suggests that the use of shape for action control may be processed independently from shape perception and, thus, like size scaling, may be refractory to the effects of crowding (Goodale, Milner, Jakobson, & Carey, 1991). In the present study, we tested this possibility and examined whether or not people could use invisible shape information to guide their actions. 

Another question that needs to be addressed is whether or not the availability of using invisible information to guide an action depends on how proficient people are in performing that action. Previous studies (Gonzalez, 2006; Gonzalez, Ganel, Whitwell, Morrissey, & Goodale, 2008) showed that grip scaling with the left hand of right-handed participants is more affected by pictorial size illusions than grip scaling with their right hand. This might be because, for right-handed participants, right-hand grasping is more skilled and, thus, more automatic and less controlled by perception than left-hand grasping. In the crowding study by Chen et al. (2015), only right-hand grasping was tested. 

Therefore, the purpose of the current study was twofold. First, we investigated whether or not people can scale their grip aperture to invisible shape information (Experiment 1). Second, we examined whether or not the proficiency in grasping matters for the use of invisible shape information (Experiment 2). To test this, left-hand grasping of right-handed participants was tested. 

Methods

Participants

Seven participants (two males, five females) took part in Experiment 1. Nine participants (four males, five females) took part in Experiment 2. All were recruited from the University of Western Ontario. They participated in the experiment in partial fulfillment of an introductory psychology course. All participants had no knowledge about the purpose of the study. They had normal or corrected-to-normal vision. Before the experiments began, all participants were required to fill out the 10-item version of the Edinburgh handedness questionnaire (Oldfield, 1971) to confirm that they were right-handed. Their ages ranged between 17 and 19 years. In both experiments, participants gave informed consent. The experiments were approved by the University of Western Ontario Ethics Review Board. 

Apparatus and stimuli

The two experiments used the same apparatus, stimuli, and procedures (Figure 1). The only difference between them was that in Experiment 1, the display was in the right visual field (Figure 2A) and participants used their right hand to perform the tasks, whereas in Experiment 2, the display was in the left visual field (Figure 2B) and participants used their left hand to perform the tasks. 

Figure 1

 

Setup and target blocks. Left, photograph of a participant performing the grasping task in Experiment 1. The black dots on her fingers are infrared light-emitting diodes. Participants wore the goggles throughout the experiment. Right, the two target blocks.

Figure 2

 

Displays of Experiment 1 (A) and 2 (B). In the uncrowded condition, a white rectangular target block was presented in isolation. In the crowded condition, the same block was surrounded by other blocks that had the same area as the target but different dimensions (flankers). (A) The display was presented in the right visual field, and participants performed the grasping and manual estimation tasks with their right hand. (B) The display was presented in the left visual field, and participants performed the grasping and manual estimation tasks with their left hand.

In both experiments, participants were seated in front of a black table with their heads stabilized by a chin rest (Figure 1, left). Participants wore liquid crystal goggles (PLATO goggles, Translucent Technologies, Toronto, ON, Canada) throughout the experiments to control the visibility of the display and their moving hand. The lenses of the goggles consist of liquid crystal cells, which are able to change rapidly from transparent to opaque, allowing an accurate control of the timing of when visual information is presented to a participant. The 3D positions of the thumb and index finger of the grasping hand (right hand in Experiment 1 or left hand in Experiment 2) were tracked with an OPTOTRAK system (Northern Digital, Waterloo, ON, Canada) in which the infrared light-emitting diodes (IREDs) were attached to the right corner of the thumbnail and the left corner of the index finger (Figure 1, left). The sample rate was 200 Hz. 

The viewing distance was 52 cm. The horizontal distance from the center of the display on the table to the fixation point was 39.5 cm (i.e., the eccentricity was 41°). A start button was located 15 cm from the edge of the tabletop facing the participants. The center of the stimulus display was directly in line with the start button (Figure 2). 

Six blocks with the same area (25 cm²) but different dimensions were used. The width of these blocks ranged from 3.5 to 4.75 cm, increasing with an interval of 0.25 cm. The thickness of the blocks was 1 cm. In both experiments, a target block was presented either in isolation (uncrowded) or surrounded by flankers (crowded). The width of the target was either 3.75 cm or 4.25 cm (Figure 1, right). The orientation of the target was 45° to the left in Experiment 1 and 45° to the right in Experiment 2. A pilot study showed that 45° is a comfortable orientation for the participants to grasp a rectangular block across its width. The remaining blocks and a copy of the target block were used as flankers, and their orientations were randomized. The distance between the center of the target and the center of a flanker was 11 cm. All flankers were fixed on the table with magnets. Only the target was movable. To make sure that participants would not worry about bumping into the flankers, the target was raised 0.5 cm higher than the flankers with a black foam pad. 

It is worth emphasizing again that all the blocks used in this study had the same surface area but different dimensions. These particular stimuli were originally devised by the American psychologist Robert Efron (1969) to diagnose individuals with visual form agnosia, who can discriminate among objects of different size but not different shape (Benson & Greenberg, 1969; Efron, 1969; Goodale et al., 1991). This suggests that, to grasp an “Efron” block accurately across one of its dimensions, the brain must process information about the shape of the object in order to extract the magnitude of the relevant dimension. In other words, grip scaling cannot be based on overall size. Thus, although our participants grasped the blocks across their width, what we tested was the effect of crowding on shape rather than size. 

Procedure and design

All participants were given 10 to 30 min of training before doing the main experiments. To make sure that participants were fixating properly, we opened the goggles before each block of trials and gave the participants time enough to adjust their head orientation so that when the goggles were opened on experimental trials, they would be looking directly ahead at the fixation point. Then we asked them to maintain fixation and keep their head still throughout the trial block even when the goggles were closed. This was practiced before they started the main experiments. We confirmed that they could remain fixated properly by asking them periodically if they had to refixate after the goggles were open. They reported no difficulty. Because they could see the fixation point before the goggles were closed and could remember the relative position of the fixation point with respect to their head, they could maintain their gaze on the fixation point reasonably well, as long as they kept their head still. The same instruction has been used in our previous study (Chen et al., 2015). The eye movement data recorded in that study when naïve participants were performing grasping and estimation tasks with the same procedures of Experiment 1 suggest that participants could maintain fixation at correct position following the same instructions during both grasping and estimation tasks (the deviation between fixation point position and actual gaze position was only 2° or so in the horizontal plane). 

At the beginning of each trial, the goggles were closed. Participants held down the start button with the thumb and index finger pinched together. After the blocks had been placed on the table, the goggles were opened. On grasping trials, participants were required to reach out and pick up the target block across its width using their thumb and index finger as quickly and as accurately as possible (grasping task). The OPTOTRAK was triggered as soon as the goggles were opened to record the entire grasping movement. On perceptual trials, participants were required to indicate the width of the target block manually by opening their thumb and index finger a matching amount (manual estimation task). When participants signalled that they were satisfied with their estimate, the experimenter triggered the OPTOTRAK to record the data. After they had made their estimate, the participants then picked up the block to ensure that on perceptual trials, they had the same haptic feedback about the width of the target as they did on grasping trials (Haffenden & Goodale, 1998). 

In both experiments, there were four combinations of conditions (2 crowding conditions [crowded vs. uncrowded] × 2 widths [3.75 cm vs. 4.25 cm]). Each combination had 10 repetitions. To prevent participants from committing the shape of the targets to memory, eight catch trials with other target widths were also included (4.5 and 4.75 cm). The presentation order of these trials was randomized for each task. The order of tasks was also randomized across participants. 

Data collection and analysis

On grasping trials, the distance between two IREDs was recorded throughout the entire movement. The PGA during the approach phase to the object was extracted (Figure 3). On perceptual trials, the distance between the IREDs on the index finger and thumb was recorded as soon as participants indicated that they were satisfied with their estimate. 

Figure 3

 

Profile of grip aperture of grasping task over time. The second platform of the profile refers to when the two fingers are holding the block. The peak grip aperture occurs well before the fingers make contact with the target.

In both experiments, the manual estimates and the PGAs were averaged for each condition and for each individual. A three-way repeated-measures analysis of variance (ANOVA) was used to analyze the main effects of crowding, task, and actual width as well as their interactions. Post hoc paired t tests (two-tailed) were also used to examine whether or not the manual estimates or PGAs of the 3.75 cm and 4.25 cm width target were significantly different. 

Results

Experiment 1

In this experiment, we examined whether invisible shape information can be used to guide grasping. Not surprisingly, when the target was presented in isolation, participants were able to distinguish between the width of the two targets with manual estimations, t(6) = 7.92, p < 0.001 (Figure 4A, uncrowded). Similarly, they also showed excellent grip scaling during grasping, t 6) = 6.35, p = 0.001 (Figure 4B, uncrowded). When the target blocks were crowded by flankers, however, participants could no longer discriminate between the two blocks using a manual estimate, t(6) = 0.35, p = 0.74 (Figure 4A, crowded), which suggests that the shape information was invisible. In contrast, when they were asked to grasp the same targets, their PGA still scaled to the width of the targets, t(6) = 9.227, p < 0.001 (Figure 4B, crowded). Because all the blocks had the same surface area but different dimensions, this suggests that information about the shape of the target can still be used to guide grasping even when that information is perceptually invisible. 

Figure 4

 

Results of Experiment 1. (A) Manual estimates in the uncrowded and crowded conditions as a function of target width. (B) Peak grip apertures in the uncrowded and crowded conditions as a function of target width. The error bars are within-subject 95% confidence intervals (Masson & Loftus, 2003).

These results were confirmed by the significant interaction between task, crowding, and width, F(1, 6) = 17.1, p < 0.001, when a three-way repeated-measures ANOVA on the manual estimates and PGAs with task (estimation vs. grasping), crowding (uncrowded vs. crowded), and target width (3.75 vs. 4.25 cm) as main factors was performed. Moreover, the interaction between crowding and width was significant for the manual estimation task, F(1, 6) = 15.89, p = 0.007, but not for the grasping task, F(1, 6) = 3.63.1, p = 0.11, which suggests that crowding interfered with manual estimation but not grasping. 

Experiment 2

In Experiment 2, we examined whether or not the effect of shape crowding on grasping depends on which hand is used (dominant or nondominant hand). To test this, we asked the right-handed participants to use their left hand, rather than their right hand, to manually estimate the width of the target block and to reach out and grasp the target block across its width. 

As was the case in Experiment 1, when the target was uncrowded, participants could distinguish the width of the target block, t(9) = 5.62, p < 0.001 (Figure 5A, uncrowded) and could also scale their grip aperture to grasp the target across its width, t(9) = 8.09, p < 0.001 (Figure 5B, uncrowded). When the target was crowded, participants could not distinguish the width of the target block, t(9) = 0.11, p = 0.92 (Figure 5A, crowded), which suggests that the shape information was invisible to them. What is more important, they could not grasp the target properly either, t(9) = 1.02, p = 0.34 (Figure 5B, crowded). This suggests that with the left hand, our right-handed participants could not use the invisible shape information to grasp objects. 

Figure 5

 

Results of Experiment 2. (A) Manual estimates in the uncrowded and crowded conditions as a function of target width. (B) Peak grip apertures (PGAs) in the uncrowded and crowded conditions as a function of target width. The error bars are within-subject 95% confidence intervals (Masson & Loftus, 2003).

Discussion

In Experiment 1, we found that grasping can to some degree escape the perceptual crowding of object shape, consistent with our previous finding that grasping is quite refractory to the perceptual crowding of object size (Chen et al., 2015). This suggests that it might be a general phenomenon that crowding has a smaller effect on real actions toward real objects than it has on perceptual reports about those same objects and that neurologically intact participants can use invisible visual information about object features to guide their actions. In Experiment 2, we found that crowding will interfere with grasping if the nondominant hand (i.e., the left hand of right handers) is used to perform the action, suggesting that the invisible visual information can be used to control an action only when that action is highly skilled. It should be noted that the words unconscious or invisible here refer to the fact that participants could not identify the shape of the target even though they were aware of its presence and position. In typical crowded arrays, the features of the target and the flankers become mixed or averaged (Parkes et al., 2001). But even so, crowding does not necessarily influence the detection of the target itself (Pelli, Palomares, & Majaj, 2004). 

Previous studies have shown that perception of the orientation of gratings with high contrast (Blake, Tadin, Sobel, Raissian, & Chong, 2006; He et al., 1996), motion (Moutoussis & Zeki, 2006), emotion (Faivre, Berthet, & Kouider, 2012; Kouider, Berthet, & Faivre, 2011), and the semantic information of words (Yeh, He, & Cavanagh, 2012) can survive crowding. The current study together with our earlier work (Chen et al., 2015) is the first to show that real actions directed to real three-dimensional (3D) objects can survive crowding. Unlike previous studies that have focused on the effects of crowding on identification or discrimination, our study focuses on the effects of crowding on action, providing a different but intriguing new perspective on how crowding might operate. As addressed in the Introduction section, there is some debate about whether crowding occurs because of the pooling of visual information at the early stages of visual processing (Chen et al., 2014; Millin, Arman, Chung, & Tjan, 2014; Nandy & Tjan, 2012) or because of poor attentional resolution of high-density displays at high-level later processing stages (Fang & He, 2008; He et al., 1996). The attention-based account suggests that the crowded information is not lost at early stages but remains available for processing by some systems but not others (i.e., by the dorsal but not the ventral stream). If this were the case, it is perhaps not surprising that the invisible shape information can be used to guide actions. The pooling account suggests that the crowded information is lost as early as V1. Even so, one might still predict that grasping would not show as much sensitivity to crowding as perceptual report. This is because, even though the ventral stream gets almost all of its input from V1, the dorsal stream gets some visual signals over pathways that bypass V1 and project instead to MT (middle temporal area; Sincich, Park, Wohlgemuth, & Horton, 2004), V3A (Girard, Salin, & Bullier, 1991), and eventually reach parieto-occipital structures, such as V6 and V6A (Colby, Gattass, Olson, & Gross, 1988). This is consistent with neuropsychological studies that showed that people with lesions in the lateral occipital cortex in the ventral stream can still grasp objects with proper grip aperture and orientation (Goodale et al., 1991; James, Culham, Humphrey, Milner, & Goodale, 2003). 

Recently, it has been shown that when flankers can be grouped together perceptually, crowding decreases with the increase in the number of flankers (Levi & Carney, 2009; Malania et al., 2007; Manassi et al., 2012; Manassi, Sayim, & Herzog, 2013; Saarela et al., 2009). For example, when a vernier is embedded in a square, Vernier-offset discrimination strongly deteriorates (i.e., crowding occurs; Manassi et al., 2013). But when more squares are added and all the squares can be grouped together, crowding disappears. This suggests that the global pattern or the stimulus configuration is critical for crowding to occur. In our study, however, the flankers were always white blocks of different widths. We did not manipulate the global pattern of the stimulus. It would be interesting in the future to examine how grouping influences the effect of crowding on action as well. 

The dissociation in the effects of crowding on perception and action that we found in the current study might at first blush seem similar to earlier illusion work (Aglioti, DeSouza, & Goodale, 1995). In that study, the investigators found that a familiar size-contrast illusion affects perception but not action. Franz, Gegenfurtner, Bülthoff, and Fahle (2000) criticized this study and argued that the perceptual and motor tasks were not well matched in terms of their attentional requirements. In the perception task used by Aglioti et al. (1995), participants were required to compare the relative sizes of the two central disks, whereas in the grasping task they grasped only one of the disks at a time. Franz and other investigators have also argued that the reason grasping, but not perception, was tuned to the real size of the object was because participants had haptic feedback on the grasping trials but not on the perception trials (Bruno & Franz, 2009; Franz, Hesse, & Kollath, 2009). In our experiments, however, we have considered all of these criticisms. Specifically, we asked the participants to manually estimate the width of one target instead of comparing the widths of two targets. In addition, participants were instructed to pick up the target after they manually estimated its width so that they had the same haptic feedback as they did in the grasping task. Therefore, the criticisms that were leveled at the Aglioti et al. (1995) study (and some other illusion experiments) do not apply to the current study on crowding. Finally, it is important to note that crowding is quite different from a size-contrast illusion. In all of the illusion studies, the target is clearly visible, even though the size might be distorted, whereas in a crowding display, the size and/or shape of the target cannot be clearly delineated, because the edges of the target are difficult to distinguish perceptually from the edges of the surrounding flankers. 

In Experiment 2, we found that unlike right-hand (dominant hand) grasping, left-hand (nondominant hand) grasping does not escape the influence of flankers in the peripheral visual field (i.e., crowding). One reason for this difference is that, for right-handers, right-hand grasping is more skilled and automatic than left-hand grasping. For example, Flindall, Doan, and Gonzalez (2014) found that right-hand grasping is faster and more accurate than left-hand grasping. Tang, Whitwell, and Goodale (2014) reported that right-hand grasping is routinely influenced by what happened on the previous trial independent of conscious knowledge, whereas left-hand grasping is affected by both the previous trial and the anticipatory knowledge about what is going to occur on an upcoming trial. 

It could perhaps be argued that the reason that left-handed grasps were influenced by crowding is due not to the hand that was used but rather the visual field in which the stimuli were located. It is interesting to note that manual estimations of the widths of the Efron blocks were as poor in the left visual field as they were in the right. Nevertheless, it is possible that targets presented in the normal workspace of the right hand (in our case, the right visual field) are processed more efficiently for action than are targets presented in the workspace of the left hand. In any case, it would appear that right handers' familiarity with grasping with the right hand in its own workspace is a potent factor in determining whether or not the visual control of action escapes the effects of crowding. 

Although we showed clear evidence that people can use shape information to guide their actions, we should note that the shapes we used are symmetrical and very simple. Further studies are required to test whether or not people can match the orientation of their grasping hand to the orientation of crowded inaccessible stimuli or place their fingers on stable grasp points of crowded inaccessible complex objects, such as those with smoothly bounded contours that lack clear symmetry (Goodale et al., 1994). 

Conclusions

In summary, we show that people can grasp objects that have different shapes accurately in cluttered scenes even when the shapes of the objects are perceptually crowded inaccessible. We also show that this striking ability depends on how skilled people are in performing that action. 

Acknowledgments

This work was funded by a grant from the Natural Sciences and Engineering Research Council of Canada (NSERC) to M.A.G. and a group training grant from the NSERC Collaborative Research and Training Experience Program (CREATE). 

Commercial relationships: none. 

Corresponding author: Juan Chen. 

E-mail: [e-mail]. 

Address: The Brain and Mind Institute, the University of Western Ontario, London, Ontario, Canada.