Cross-modal effects of auditory magnitude on visually guided grasping

Abstract

Recent research has established the role of objects' semantic properties in the planning of motor actions with respect to these objects. It has been shown that visual numerical magnitude affects visuomotor control in a similar direction to the effect of physical size: The larger the numerical value, the larger the grip aperture even when physical size remains invariant. The relationship has been attributed to a common mechanism, in particular to a neural network within the parietal lobe, which mediates the processing of magnitude across different domains. In this study, we show that the effect of magnitude on grasping is not limited to visual numerical information and is in fact cross-modal in nature; presentations of auditory signals of different types of auditory-based magnitudes affected visually guided actions in two different experiments. In Experiment 1, symbolic representations of magnitudes (numerals) affected initial grasping trajectories. In Experiment 2, a nonsymbolic presentation of magnitude, i.e., tone duration, had similar effects on grasping trajectories. We conclude that different types of magnitude representations are processed by a common mechanism that cooperates with visuomotor control.

Introduction

Reaching for a tennis ball on the ground, people separate their fingers during motion and prior to the actual grasping to an extent that matches the size of the ball. This motor preparation for action is remarkably accurate—the larger the graspable object the larger the aperture—and it is probably performed in an automatic fashion (Bub & Masson, 2010; Ganel, Freud, Chajut, & Algom, 2012; Jakobson & Goodale, 1991). Motor cortical regions are activated in an instantaneous fashion even when people view graspable objects (Martin, Haxby, Lalonde, Wiggs, & Ungerleider, 1995; Martin, Wiggs, Ungerleider, & Haxby, 1996; Sumner & Husain, 2008). Consequently, mental representations for action are afforded in advance, preparing for handling the object of interest. An intriguing question concerns the symbolic size of objects. Consider the numeral 8. Apart from its physical (font) size, this stimulus carries symbolic size, too (e.g., “large” if the set of numbers are in the single-digit range). Does symbolic size affect initial grasping trajectories in the same way as physical size? Are numerically larger numerals conductive to larger apertures than numerically smaller digits? Despite evidence in the affirmative (Andres, Ostry, Nicol, & Paus, 2008; see also Glover, Rosenbaum, Graham, & Dixon, 2004) the issue is not fully settled. In particular, the precise association between symbolic and nonsymbolic magnitude and grasping is still missing. Filling this lacuna was the goal of the present study. 

Support for semantic involvement in action comprehension was provided in the study by Campanella and Shallice (2011) who found that it was harder to distinguish between a pair of objects requiring similar hand actions than between a pair of objects sharing only visual similarity. Inevitably, semantic properties associated with graspable objects in view are encoded and activated in an automatic fashion. Further striking evidence for semantic activation in prepared action is provided in the recent study by Bub, Masson, and Lin (2013). These authors created congruent and incongruent displays by combining hand-action components such that the orientation of the object's handle (vertical, horizontal) matched or mismatched the orientation of the wrist or that the alignment of the object's handle (right, left) matched or mismatched the hand's side of the body. The authors found that congruency affected object naming in a profound way such that the planning of actions disrupted the ability to name the objects. For example, planning left-handed actions interfered with naming a beer mug whose handle aligned with the right hand. Notably, the effect of planned action extended to imagery and memory, too. 

The information provided by these studies is singularly valuable, yet all tested the effect of semantic knowledge on action affordances or perception of action (and vice versa). However, the perception of objects and the actual motion directed toward the same objects are subserved by dissociable neural and cognitive mechanisms (Ganel & Goodale, 2003; Milner & Goodale, 2008). Therefore, the goal of the present study was to pursue the dynamics of visuomotor control with respect to different magnitude representations during grasping. 

Walsh (2003) has famously proposed that the processing of magnitude, space, time, and quantity are shared by a set of cortical areas within the parietal lobe. An overlapping set of areas is thought to mediate online visuomotor control, including reaching and grasping (Cavina-Pratesi et al., 2010). After all, visuomotor control requires precise integration of magnitude-based spatial and temporal information. Support for the hypothesis of a kinship between visuomotor control and (symbolic) magnitude has been recorded in studies showing increased activation in the intra parietal sulcus (IPS) when processing numerical magnitude (Kadosh et al., 2005) and when performing object-directed actions (Culham, Cavina-Pratesi, & Singhal, 2006; Frey, Vinton, Norlund, & Grafton, 2005). Movement preparation was also found to be associated with adjacent cortical areas (i.e., in the inferior and superior parietal lobule, Astafiev et al., 2003). 

Several studies have tested the contingency of visuomotor control on symbolic magnitude in a more direct fashion. An impressive relationship was reported. For example, size-related words were shown to affect grip aperture, with words describing large objects (e.g., “apple”) leading to larger grip apertures than words describing smaller object (e.g., “grape,” Glover et al., 2004). In a similar vein, numerically larger digits resulted in faster reaction times when participants were asked to use power-grip grasping (grasping objects using their full hand grip) (Moretto & Pellegrino, 2008). In a related study, Badets, Andres, Di Luca, and Pesenti, (2007) showed that numerical magnitude also influences people's subjective assessment of their ability to perform actions. In particular, presenting numerically larger digits resulted in subjective underestimation of grasping ability; another study by Lindemann, Abolafia, Girardi, and Bekkering (2007) showed how numerical magnitude affects both reaction times for different types of grasping and the maximum grip aperture while grasping. 

Although the role of symbolic magnitude in grasping is firmly established (as well as that of the generic involvement of semantic analysis in action planning), the precise substrates that mediate the relationship are less clearly delineated. Impressed with the powerful contingency, many elect to assume a common underlying mechanism. For example, Bub et al. (2013) suggested that “a common representational format underlies the processing of perceived objects and planned actions” (pp. 6–7). Similarly, Walsh (2003) assumes a common or a largely shared neural mechanism. What is missing in these proposals is fine experimental detail. How is exactly grasping trajectory evolves as function of magnitude? Does every kind of magnitude processing affect grasping equally or even minimally? A negative answer would cast doubt on the validity of the hypothesis on a fully shared underlying source, whereas a positive will provide a compelling support for these claims. 

Several studies found that numerical magnitude influences grasping primarily at the initial stages of the movement trajectory, but it has no influence on grasping trajectories at later stages of movement (i.e., when the fingers approach the target object). For example, Fischer and Miller (2008) showed that numerical information affects the RT to initiate button-press responses but has a weaker effect on the applied force of the final response. These authors suggested that magnitude affects the planning of the movement rather than the actual performance (see also Badets et al., 2007; Ishihara et al., 2006; Lindemann et al., 2007). In another study by Andres, Ostry, Nicol, and Paus (2008), large (8–9) or small (1–2) digits were shown embedded in rectangular objects. The participants were asked to pick the object and place it at the appropriate location defined by the digit's parity. The results showed that, during initial stages of the movement trajectory (at about 10%–40% of the movement), the fingers opened larger in response to the numerically larger digits. Recently, we have also provided evidence that the effects of visual numerical magnitude on grasping can be considered automatic in nature (Moretto & Pellegrino, 2008; Namdar, Tzelgov, Algom, & Ganel, 2014). In particular, grasping trajectories at initial movement stages were affected by numerical magnitude even when participants were asked to attend to a magnitude-irrelevant dimension of the digit's color rather than to its numerical value. By contrast, no effects of numerical magnitude were found during later trajectories of the movement. 

Compared to the relatively established evidence for the effect of visually based magnitude information on action, it is still not clear whether magnitude information transmitted by different modalities other than vision invokes similar effects on visuomotor behavior. However, some evidence is found that different aspects of magnitude of auditory signals share the same common metric for magnitude as visual cues. Javadi and Aichelburg (2012) asked participants to judge either numerosity or presentation duration of a set of items; they found that larger sets were judged as being presented for longer durations and that sets presented for a longer duration were judged as larger in magnitude, thus implying a somewhat general representation of magnitude. Further evidence for a common metric for magnitude arises from a study performed by Crollen, Grade, Pesenti, and Dormal (2013) in which participants were required to numerically estimate numerosity, length, and duration of a stimuli or to produce a stimuli that matches a number. A similar pattern of results was found across all types of magnitudes, suggesting, again, the possibility of a common underlying mechanism for magnitude processing. In a similar fashion, the work of Heinemann, Pfister, and Janczyk (2013) has recently showed that tones with different intensities and durations can affect random number generation, suggesting that auditory magnitude processing may be interconnected with numerically based processing. 

In the current study, we tested whether magnitude affects grasping in a cross-modal fashion. In particular, we tested for the effect of auditory-presented digits and of nonsymbolic aspects of auditory tones on grasping trajectories when participants are asked to grasp neutral objects. This design was used here to test the strong version of the idea according to which a shared underlying system processes magnitude across a wide variety of magnitude modalities, all related to visuomotor control. 

Several previous studies have looked at the possibility of cross-modal effects of irrelevant object size during grasping. For example, Castiello, Zucco, Parma, Ansuini, and Tirindelli (2006) tested the effects of smells of vegetables and fruits odors (e.g., the odor of a strawberry) on grasping kinematics toward plastic replicas of these vegetables and fruits (e.g., grasping an apple). The results showed congruency and incongruency effects of smell (e.g., the smell of small objects such as a strawberry or an almond) on grasping kinematics toward objects (e.g., grasping large objects such as an apple or an orange). In a similar vein, Parma, Ghirardello, Tirindelli, and Castiello (2011) tested the effects of flavor congruency on grasping, and Castiello, Giordano, Begliomini, Ansuini, and Grassi (2010) tested the effects of contact sound congruency on grasping (for similar studies, also see Patchay, Castiello, & Haggard, 2003; Tubaldi, Ansuini, Tirindelli, & Castiello, 2008). In most of the experiments in these previous studies, the congruency effects were evident during Maximum Grip Apertures (MGAs), the maximum opening between the fingers that occurs at the second part of the movement trajectory. More importantly, in all of these studies, the irrelevant cross-modal effects were always associated with memory-representations of stored motor actions toward known objects (e.g., a stored motor plan to grasp a strawberry). This makes it difficult to disentangle the effects of previous motor plans toward specific objects (e.g., strawberries or apples) from effects of their symbolic sizes. Therefore, it can be argued that the cross-modal interference effects reported in previous studies have resulted from discrepancies between learned skill motor plans and between online grasping kinematics rather than from the effect of magnitude per se. To that extent, our study uses more general representations of magnitude (numeric magnitude in Experiment 1, or nonsymbolic, tone duration magnitude in Experiment 2), relatively free of specific associations with learned motor programs, making it possible to argue that any cross-modal effects we observe in Experiments 1 and 2 could be attributed to influences of magnitude per se rather than to incongruency effects with stored motor plans. 

The goal of the present study therefore was to further learn about the relationship between visuomotor control and magnitude processing in a detailed, experimentally rigorous manner. In Experiment 1, we tested the relationship between movement planning and nonvisual magnitude processing. We examined the effect of auditory-presented, symbolic magnitude information, on grasping trajectories. To the extent that action planning and numerical magnitude processing are mediated by a common underlying mechanism, the effect of magnitude would be apparent across modalities. In Experiment 2, we tested whether nonsymbolic representation of magnitude (i.e., tone duration) would have similar effects on grasping. 

Experiment 1

Methods

Participants

A group of 21 right-handed students from Ben-Gurion University of the Negev, who gave their consent, performed in the experiment (seven males; mean age 24.2, SD = 1.79 years). 

Stimuli and procedure

The participants sat in front of a black tabletop with their index finger and thumb touching on a small red button that served as a starting point. Participants wore a set of LCD glasses (Translucent Technologies, Toronto, ON) with liquid-crystal shutter lenses that were used to control stimulus exposure duration. Stimuli consisted of three white wooden blocks (50/55/60 mm long, 17 mm high, and 30 mm wide). In each trial, one stimulus was placed in a fixed position, 25 cm in front of the participant. Each trial began with a prerecorded auditory male voice, reading one of four possible digits (1, 2, 8, and 9), followed by the opening of the glasses for 2000 ms. During that time, participants were instructed to grasp the object and then place it in one of two separate locations on the tabletop, 10 cm to the left or to the right of the object's initial location, based on the parity of the digit that they have heard. The association between parity and location was counterbalanced between participants. As in previous studies (Andres et al., 2008; Namdar et al, 2014), the values of the smaller (1, 2) and the larger (8, 9) digits were later collapsed for the analysis and were defined as small and large magnitude, respectively. The experiment consisted of one experimental block in which every combination of stimulus size and digit was presented 12 times, making for 144 trials in all. 

Kinematic recordings and analysis

An Optotrak Certus device (Northern Digital, Waterloo, ON) was used to track the participants' hand and fingers grasping trajectories. Three infra-red light emitting didoes were placed on the right hand's thumb, index finger (located on top the center of the distal phalanges), and wrist of each subject. The wrist marker served only as potential source of data for future analysis of wrist velocity. The data collected from the wrist marker was not used in the analysis. We collected data about the velocity, acceleration, and the aperture between the index finger and thumb throughout the entire grasping movement (at a sampling rate of 200 Hz). Movement onset was determined at the point in time in which the aperture increment between the index finger and the thumb was above 0.1 mm for at least 10 successive frames (50 ms). Movement offset was determined at the first point in time in which the aperture between the index finger and the thumb changed by less than 0.1 mm for at least 10 successive frames (50 ms). 

Trials were excluded from the analysis if (a) the participant made an error by placing the object in the wrong location after grasping it, if (b) the IREDs diodes placed on the fingers were not visible to the camera while moving toward the object, and if (c) the participant failed to grasp the object properly and dropped it while trying to lift it off the table. Overall, 6.1% of the trials were excluded for those reasons. 

Results and discussion

Kinematic responses were analyzed using two different methods. First, we conducted a traditional kinematic analysis looking at the aperture between the finger and thumb throughout the grasping trajectory. In order to test the effect of digit magnitude on grip aperture, we divided each trial (for each participant) into 11 movement segments equal in length (0% to 100%). This allowed measuring the effect of digit magnitude on grip aperture in each of these 11 points in time, by deducting the average grip aperture of the low magnitude condition, 1 and 2, from the high magnitude condition, 8 and 9 (see Figure 1). 

Figure 1

 

Average differences in grip aperture between large and small numerical magnitudes along normalized movement trajectories following an auditory presentation of numerals. Note that the effect of magnitude is evident only at early parts of the movement trajectory. Error bars represent 95% confidence intervals of the interaction between movement and magnitude (Jarmasz & Hollands, 2009).

A glimpse at Figure 1 reveals that the effect of digit magnitude on aperture was largest at the first segment of the movement (at about the first third which closely replicates earlier findings (Andres et al., 2008). We analyzed the data using a repeated-measures ANOVA of digit magnitude (small/large) × object size (50/55/60 mm) × movement (0%–100%). Main effects were found for size, F(2, 40) = 140.83, p < 0.001, η_p² = 0.87; movement, F(10, 200) = 523.32, p < 0.001, η_p² = 0.96; and magnitude, F(1, 20) = 7.58, p = 0.012, η_p² = 0.27. A significant movement × magnitude interaction, F(10, 200) = 1.89, p = 0.048, η_p² = 0.08, indicated that magnitude had differential effects on grip aperture during different stages of the movement. Further analysis revealed a significant effect for magnitude over the first part (10%–50%) of the movement, F(1, 20) = 10.81, p = 0.003, η_p² = 0.35, but not for the second part (60%–100%) of the movement, F(1, 20) = 1.56, p = 0.22, η_p² = 0.07. Given that a similar pattern of results were found in previous studies (Andres et al., 2008; Namdar et al., 2014), we conducted a dedicated planned comparison that revealed that the difference between the effects of magnitude on the first part (10%–50%) and the second part (60%–100%) of the movement was significant, F(1, 20) = 3.882, p = 0.031, η_p² = 0.16, one tailed. Figure 2 plots the effect of magnitude at the first and second parts of the movements for each individual subject. 

Figure 2

 

The difference in the effect of magnitude on grip aperture between the first and second parts of the movement for each individual subject. Each dot represents a single participant. As can be seen in the figure, for most subjects, the effect of magnitude was larger in the first part of the movement, which is consistent with the grand average data presented in Figure 1.

To complement the analysis of the effects of magnitude at different stages of the movement, we entered the MGA data to a repeated-measures ANOVA of 3 (object size) × 2 (digit magnitude) as independent variables. A main effect was found for size, F(2, 40) = 179.4, p = 0.0001, η_p² = 0.89, with larger MGAs for the larger objects (84.2, 86.78, and 89.4 mm for the 50, 55, and 60 mm long objects, respectively). Although MGAs were larger for the high compared to the low digit magnitude (86.92 mm and 86.67, respectively), the main effect of magnitude was not significant, F(1, 20) = 2.94, p = 0.10, η_p² = 0.12. The interaction between object size and magnitude as well was not significant, F(2, 40) = 1.5, p = 0.23, η_p² = 0.07. 

To rule out the possibility of speed-accuracy tradeoffs, we analyzed movement times in a similar fashion to the grip aperture analysis. Movement times were entered into a repeated- measures ANOVA of 3 (object size) × 2 (digit magnitude) within subject variables. A significant effect was found for size, F(2, 40) = 5.17, p = 0.01, η_p² = 0.20, with slower movement times for smaller compared to bigger objects: 655 (91) ms, 650 (93) ms, and 644 (83) ms for the 50 mm, 55 mm, and 60 mm objects, respectively. The main effect of digit magnitude was not significant, F(1, 20) < 1, p = 0.34, η_p² = 0.04, ruling out possible alternative accounts according to which the effect of magnitude information on grasping was mediated by speed-accuracy tradeoff (faster movement times in the high magnitude condition). The interaction between object size and digit magnitude was not significant, F(2, 40) = 1.92, p = 0.15, η_p² = 0.08. 

The results of Experiment 1 provide new insights on to the relationship between magnitude processing and visuomotor control. The findings show that the effect of numerical magnitude on grasping is robust, and transforms across different modalities. In particular, magnitude information affects visually guided motion even when it is presented via auditory channels. This result lends further support to the idea that magnitude information is stored in a general fashion and is processed within a single underlying magnitude system. 

Experiment 2 was designed to provide an even stronger test for this idea. To this purpose, we tested whether nonsymbolic auditory magnitude information would affect grasping in a similar manner to the effect of symbolic information embedded in numerals. In Experiment 2, participants were presented with auditory tones varying in length (and thus in their perceived duration). We reasoned that, if indeed magnitude information is stored and processed within a common underlying system, its effects on grasping should be visible not only across different modalities (e.g., auditory-visual interactions) but even across different types of representations (e.g., symbolic and nonsymbolic magnitudes). 

Experiment 2

Methods

Participants

Twenty-four right-handed students from Ben-Gurion University that gave their consent to take part in Experiment 2 (six males; mean age 23.58 years, SD = 1.6). All subjects had normal or corrected to normal vision. Participants received $7.00 for their participation. 

Stimuli and procedure

In Experiment 2, we used the same basic setup (table, stimuli, and glasses) used in Experiment 1. However, instead of numerals, magnitude information was conveyed using a 440 Hz tone varying in duration (500 ms/2000 ms). The tone was sounded from one of two speakers located at the far corner of a table and in 45° from the participant's midline. One speaker was located in the right corner, and a second one was located in the left corner of the table. Vision was allowed only after the tone has ended. Participants were instructed to grasp the object and to then place it in predesignated location corresponding with the side from which the tone has been originated (and irrespective of the tone's duration). Each combination of sound direction, object size, and tone duration was repeated 12 times (in a pseudorandomized order), resulting in a total of 144 trials. 

Kinematic recordings

The kinematic recording procedure was similar to that of Experiment 1. Trials were excluded from the analysis using the same criteria as in Experiment 1, resulting in the exclusion of 6.3% of the trials. 

Results and discussion

We entered the grip aperture data into an ANOVA with movement (0%–100%), sound direction (left/right), object size (50, 55, and 60 mm) and duration (500 ms vs. 2000 ms) as independent variables. A significant interaction was found between movement and duration, F(10, 230) = 3.66, p < 0.0001, η_p² = 0.13, indicating that the tone duration had differential effects on grip aperture during the movement trajectory. The main effects of object size, F(2, 46) = 110.2, p < 0.0001, η_p² = 0.82, movement, F(10, 230) = 466.8, p < 0.0001, η_p² = 0.95, and duration, F(1, 23) = 8.18, p < 0.01, η_p² = 0.26, were also significant. No significant main effect was found for sound direction, F(1, 23) = 3.4, p > 0.05. A significant interaction was found between movement and sound direction, F(10, 230) = 2.34, p < 0.02, η_p² = 0.09. A more detailed examination of the source of the interaction revealed that it originated from the fact that when the tone stimuli came from the right direction, the aperture between the thumb and index finger was slightly larger during a specific part of the movement (40% to 70% of the normalized movement). 

More importantly, as can be seen in Figure 3, the largest effect of tone duration on grip aperture was evident at the first half (10%–50%) of the movement trajectory, in a similar fashion to the effects of numerical magnitude found in Experiment 1 and in previous studies. Planned comparisons revealed a significant effect for tone duration on grip aperture in the first part (10%–50%) of the movement, F(1, 23) = 10.07, p = 0.004, η_p² = 0.30, but as in Experiment 1, not in the second part (60%–100%) of the movement, F(1, 23) = 2.40, p = 0.13, η_p² = 0.09. An additional analysis revealed a significant difference between the first part (10%–50%) and the second part (50%–100%) of the movement, F(1, 23) = 6.32, p = 0.019, η_p² = 0.21. Figure 4 plots the difference between the effect of magnitude at the first and second parts of the movements for each individual subject. 

Figure 3

 

Average differences in grip aperture between long and short tone durations along normalized time trajectories of the movement in Experiment 2. Longer tone durations resulted in a larger grip aperture compared to shorter durations. As in Experiment 1, this effect was pronounced mostly at early parts of the movement trajectory. Error bars represent 95% confidence intervals of the interaction between movement and magnitude (Jarmasz & Hollands, 2009).

Figure 4

 

The difference in the effect of magnitude on grip aperture between the first and second parts of the movement for each individual subject. Each dot represents a single participant. As in Experiment 1, for most subjects, the effect of magnitude was larger in the first compared to the second part of the movement.

As in Experiment 1, we entered the MGA data to a repeated-measures ANOVA of 2 (sound direction) × 3 (object size) × 2 (tone duration) as independent variables. A main effect was found for sound direction, F(1, 23) = 9.08, p = 0.006, η_p² = 0.28, with larger MGAs following the presentation sounds from the right (86.2 mm) compared to the left (86.76 mm) side of the participants. A main effect was also found for size, F(2, 46) = 95.6, p = 0.0001, η_p² = 0.80 with larger MGAs for larger objects (83.51, 86.13, and 88.3 mm for the 50, 55, and 60 mm long objects, respectively). More importantly, no main effect was found for tone duration, F(1, 23) = 1.81, p = 0.19, η_p² = 0.07, with average MGAs of 85.84 mm for the shorter tone and 86.12 mm for the longer tone. All interactions were not significant except for an unpredicted interaction between object size and tone duration, F(2, 46) = 3.52, p = 0.03, η_p² = 0.13. This interaction originated from relatively larger difference in MGA between the long and the short tone durations for the 50 mm object compared to the other objects. 

As in Experiment 1, to rule out a possible effect of speed accuracy tradeoff, we entered the movement times (MT) to a repeated-measures ANOVA of 2 (sound direction) × 3 (object size) × 2 (duration) within-subject variables. A significant effect was found for sound direction, F(1, 23) = 5.81, p = 0.02, η_p² = 0.20, with slower MTs for the right side (602 ms) compared to the left side (598 ms) condition. A significant effect was also found for object size, F(2, 46) = 4.33, p = 0.01, η_p² = 0.15, with slower MTs for smaller compared to bigger objects (605 ms, 600 ms, and 596 ms for the 50 mm, 55 mm, and 60 mm objects, respectively). Finally, a significant effect was found for tone duration, F(1, 23) = 7.83, p = 0.10, η_p² = 0.25, with an average MT of 607 ms and 593 ms for the long duration and the short duration, respectively. The finding that MTs were slower, not faster, for the long compared to the short duration ruled out possible effect of speed-accuracy tradeoff. A significant interaction was also found between sound direction and duration, F(1, 23) = 4.88, p = 0.03, η_p² = 0.17, with 8 ms slower MTs in the right side compared to the left side for the low magnitude condition, accompanied by a 2 ms corresponding difference for high magnitudes. All other interactions were not significant (all Fs are lower than 1). 

In Experiment 2 we instructed participants to grasp a neutral object following an auditory presentation of tone of different durations. The results converge with those found in Experiment 1 to support the hypothesis that nonsymbolic magnitude information affects grasping trajectories across modalities (via the auditory channel) as in the case of symbolic magnitude information. 

General discussion

The purpose of the current study was to further examine the relationship between the processing of magnitude-related information and visuomotor control. To this purpose, we looked at the effect of auditory magnitude-based information on fingers kinematics during grasping. In Experiment 1, auditory presentations of numerals preceded each neutral grasp. The results showed a clear effect of magnitude on grip apertures during early stages of grasping, with large numbers leading to larger apertures compared to smaller numbers. These findings suggest that a symbolic representation of magnitude (i.e., numerals) can affect movement in a cross-modal fashion. We further tested this hypothesis in Experiment 2 by manipulating the way in which the information about magnitude was conveyed. In particular, single tones varying in duration preceded each grasp. We reasoned that to the extent that visuomotor control and magnitude processing are subserved by a common underlying system, magnitude information across different domains would have similar effects on movement control. The results of Experiment 2 converged with those found in Experiment 1, showing similar effects of magnitude on grip aperture, mostly during initial stages of the grasp. 

Magnitude effects at different stages of the grasping movement

Previous studies have shown that the effects of magnitude on grasping are evident at initial but not in later trajectories of the movement. This evidence was interpreted to suggest that these effects can be attributed to planning stages of grasping (Andres et al., 2008; Namdar et al., 2014). Note, that there have been earlier suggestions in the literature that at later stages of the movement, when the fingers approach the target object, visual feedback from the fingers and from the target object work to attenuate the early effect of action planning (Glover & Dixon, 2001). However, more recent evidence, including evidence provided by Dixon and Glover themselves, does not support this model. There are many examples in the literature that visual feedback does not abolish different types of task-irrelevant effects during grasping, which are evident in later movement stages (e.g., Dixon & Glover, 2009; Dixon, McAnsh, & Read, 2012; Freud & Ganel, 2015; Glover & Dixon, 2013; Holmes & Heath, 2013; Tang, Whitwell, & Goodale, 2015). Moreover, Dixon and Glover (2009) showed that repetition effects of previous motor actions influence the control of grasping even at late stages of the movement and regardless of whether visual feedback is provided or not. Based on these and on other relevant findings, the former theoretical distinction according to which the effects of motor planning in later movement trajectories are attenuated by visual feedback has been revisited. According to more recent model of movement control (Dixon & Glover, 2009; Dixon et al., 2012; Glover & Dixon, 2013), the specific attributes of motor planning toward an object affect movement trajectories at initial movement stages (i.e., the first part of the movement). However, the memory of a specific programming of an action can influence grasping trajectories throughout the entire movement, even at later stages of the movement when the fingers approach the target object (Dixon et al., 2012). A detailed analysis of the effect of magnitude throughout the movement and on the maximum grip aperture is therefore valuable to learn about the nature of the interaction between magnitude processing and action control. Indeed, the present findings suggest that the cross-modal effect of magnitude we observe can be attributed to the stage of motor planning rather than reflecting a reactivation of a stored memory program of a previous grasp. 

It is intriguing to speculate as to the nature of the effects observed in previous studies that looked at cross-modal effects of stimulus size on grasping in terms of the movement times. The congruency effects in those studies were evident during both early and late stages of the movement trajectories. First, note that vision was allowed in these studies, and that the findings of congruency effect even under full vision in late movement trajectories is clearly not in line with a simple account of visual feedback, according to which irrelevant effects on grasping are always attenuated at late stages of the movement due to visual feedback from the fingers and the target object (Glover & Dixon, 2001). It is even more interesting to discuss the results of Castiello et al. (2006, 2010), of Parma et al. (2011) and of Patchay et al. (2003) in terms of Glover and Dixon's more recent model of action planning and control (Dixon & Glover, 2009; Dixon et al., 2012; Glover & Dixon, 2013). According to this model, effects of memory-based previous motor plans are persistent throughout the entire movement trajectory, including during late stages of the movement in which MGAs occur. Therefore, it can be argued that the fact that congruency effects in previous studies were evident throughout the entire movement is due to the fact that the control of movement in these studies relied on stored memory representations of skilled actions. 

Conclusions

The results of the two experiments suggest that movement planning and magnitude processing are subserved by an interconnected functional and neuroanatomical system. These findings invite future research that would examine the relationship between visually guided actions and different aspects of object-magnitude associations. Such a line of investigation could potentially advance the understanding of the way different brain systems that mediate action and perception interact with one another. 

Acknowledgments

We thank Daniel Algom for his help and advice. This study was supported by an Israel Science Foundation (ISF) grant 640/12 to Tzvi Ganel and to Daniel Algom. 

Commercial relationships: none 

Corresponding author: Tzvi Ganel.