Open Access
Article  |   September 2021
Perceptual influence of auditory pitch on motion speed
Author Affiliations
  • Gangsheng Zhang
    Graduate School, Air Force Engineering University, Xi'an, China
    [email protected]
  • Wei Wang
    Air and Missile Defense College, Air Force Engineering University, Xi'an, China
    [email protected]
  • Jue Qu
    Air and Missile Defense College, Air Force Engineering University, Xi'an, China
    [email protected]
  • Hengwei Li
    Graduate School, Air Force Engineering University, Xi'an, China
    [email protected]
  • Xincheng Song
    Graduate School, Air Force Engineering University, Xi'an, China
    [email protected]
  • Qingli Wang
    Air and Missile Defense College, Air Force Engineering University, Xi'an, China
    [email protected]
Journal of Vision September 2021, Vol.21, 11. doi:https://doi.org/10.1167/jov.21.10.11
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Gangsheng Zhang, Wei Wang, Jue Qu, Hengwei Li, Xincheng Song, Qingli Wang; Perceptual influence of auditory pitch on motion speed. Journal of Vision 2021;21(10):11. https://doi.org/10.1167/jov.21.10.11.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

There is a cross-modal mapping between auditory pitch and many visual properties, but the relationship between auditory pitch and motion speed is unexplored. In this article, the ball and baffle are used as the research objects, and an object collision experiment is used to explore the perceptual influence of auditory pitch on motion speed. Since cross-modal mapping can influence perceptual experience, this article also explores the influence of auditory pitch on action measures. In Experiment 1, 12 participants attempted to release a baffle to block a falling ball on the basis of speed judgment, and after each trial, they were asked to rate the speed of the ball. The speed score and baffle release time were recorded and used for analysis of variance. Since making explicit judgments about speed can alter the processing of visual paths, another group of participants in Experiment 2 completed the experiment without making explicit judgments about speed. Our results show that there is a cross-modal mapping between auditory pitch and motion speed, and high or low tones cause perception shift to faster or slower speeds.

Introduction
When people experience external events, their sensory system receives multiple inputs and transmits them to the perceptual system, which then integrates these inputs for disambiguation. In this process, spatial and temporal information is usually the key to information integration (Calvert et al., 2004; Frens et al., 1995; Jones & Jarick, 2006; Shore et al., 2006; Slutsky & Recanzone, 2001; Spence & Driver, 2004; van Wassenhove et al., 2007), but cross-modal mapping between different senses also contributes to information integration. A large number of researches now show that many nonarbitrary associations appear to exist between many different basic physical stimulus attributes or features in different sensory modalities. For example, in synesthetes, a color may evoke a smell (Gilbert et al., 1996; Kemp & Gilbert, 1997), a visual shape may evoke an additional taste (Gal et al., 2007), a pitch might produce an additional sensation of smell (Belkin et al., 1997), and so on. 
Pitch is involved in a lot of cross-modal correspondences, which is the auditory feature that we investigated. In speeded reaction tasks, participants responded more quickly to the written word “high” in the presence of a high pitch and more quickly to the written word “low” in the presence of a low pitch (Melara & Marks, 1990), suggesting that the association between linguistic labels of spatial location and pitch is not arbitrary. In addition to its association with linguistic labels of spatial location, pitch also affects the speed of visual target location recognition: In a series of rapid categorization tasks, participants responded faster to a high visual stimulus in the high-pitched condition and responded faster to a low visual stimulus in the low-pitched condition (Evans & Treisman, 2010; Patching & Quinlan, 2002). When horizontal responses were required, compared to nonmusicians, musicians showed faster and more accurate horizontal associations in response to low-pitched tones on the left and to high-pitched tones on the right (Lidji et al., 2007; Rusconi et al., 2006). 
The brightness of the tone's timbre might contribute to spatial association. Pitteri et al. (2017) investigated the spatial musical association of response codes (SMARC) effect in a group of nonmusicians. The results suggest that the SMARC effect is due to a coherent change of pitch-height and brightness, and the effect emerges along the vertical axis only. Soon after, He (Pitteri et al., 2021) studied the SMARC effect with musicians as subjects and compared it with previous results. The latest results showed that the coherent modulation of both pitch-height and brightness elicited the strongest SMARC effect, independently of music expertise. 
The loudness of tones is often addressed spatially in Western languages. Bruzzi et al. (2017) investigated whether loudness might also have a spatial representation and found that participants were faster in a situation where they pressed the key at the top to report louder sounds and the key at the bottom to report quieter sounds. The result supported the view that loudness might be represented spatially. Moreover, visual motion makes sounds louder (Maniglia et al., 2017). When auditory stimuli were of the same intensity, participants judged the sound accompanied by the moving disc as louder. The effect was still present for mid-to-high intensities. Moreover, the effect on pitch was reversed compared to the observed loudness, with mid-to-high frequency sound accompanied by motion rated as lower in pitch with respect to the static intervals. 
The pitch vertical correspondence was found also in children (Nava et al., 2016), with an adult-like sensitivity to the correspondences still developing by age 5 years. Moreover, spatial ability can be developed. The researchers found that both pitch and numbers activate parietal regions (Sandrini et al., 2004; Schmithorst & Holland, 2003) and share similar spatial mental representations (Keus et al., 2005). This suggests that pitch training can improve spatial ability (Brochard et al., 2004) and be used for mathematics achievement (Cheek & Smith, 1999). 
In addition to corresponding spatial position, pitch also matches other visual attributes. One is the lightness and darkness of a surface. Participants matched a higher pitch to a lighter surface and a lower pitch to a darker surface, responding more quickly to a lighter surface with a higher pitch and a darker surface with a lower pitch (Marks, 1987; Martino & Marks, 1999; Melara, 1989). Another is shape. Marks (1987) designed two visual stimuli (an upturned “V” and an upturned “U”) to explore the impact of pitch/shape on processing human information and found that participants would match a higher pitch with the more angular shape and a lower pitch with the more rounded shape. In addition, pitch matches size. When determining the size of the second variable-sized disk, participants responded significantly faster and more accurately in congruent cross-modal trials (such as a high pitch and a small disk), demonstrating that a task-irrelevant pitch can significantly influence participants' responses in a rapid visual size discrimination task (Gallace & Spence, 2006). The results were similar in other studies (Grassi, 2005; Moore, 1977). Evans and Treisman (2010) designed nine speeded classification paradigms to study four cross-modal mappings of auditory pitch to visual position, size, spatial frequency, and contrast and found spontaneous cross-modal correspondence between auditory pitch and vertical position, size, and spatial frequency features but not contrast. 
Although researchers have generally adopted different experimental approaches, a large number of researches have converged on the conclusion that there are many nonarbitrary cross-modal correspondences between various pitch and visual stimulus features. However, the relationship between pitch and motion speed is rarely discussed in these studies. Whether there is a cross-modal correspondence between pitch and motion speed is the focus of our research. Pitch is mainly determined by frequency (Grassi et al., 2013). In real life, when a car is driving, its engine usually makes a sound, and the faster the car goes, the faster the sound frequency increases. The same is true for insects: Other things being equal, the frequency with which an insect flaps its wings is proportional to its flight speed. The faster the insect flaps its wings, the higher the pitch it emits, and the faster it flies. These phenomena mean that different tones of pitch may alter people's perception of the speed of movement. We hypothesize that there is a relationship between pitch and speed and that this relationship affects people's perception of speed. So when the target is moving at the same speed, people may make different judgments of how fast the target is moving in the presence of different tones. 
Previous studies have shown that cross-modal mapping between pitch and visual properties may occur automatically and affect specific behaviors when participants make associations (Spence, 2011). Therefore, we are also concerned about whether the relationship between pitch and motion speed will affect behavioral measures. The question of whether the association between pitch and motion speed affects specific behavioral measures or only cognitive measures depends on the difference between cognitive and behavioral measures. Researchers have devised several methods to reveal the difference between cognitive and behavioral measures. Two of the more effective methods apply visual illusions. One method uses the Ebbinghaus illusion: Two circles of the same size are placed on a picture, one surrounded by some larger circles and the other surrounded by some smaller circles; the circle inside the larger circles will look smaller than the circle inside the smaller circles. Haffenden and Goodale (1998) experimented with this illusion, in which participants were required to indicate the apparent size of a circle or pick it up. Their results showed that the Ebbinghaus illusion was reflected in size estimation rather than gripping. Furthermore, the Ebbinghaus illusion has been shown to be limited (Brenner & Smeets, 1996; Daprati & Gentilucci, 1997; Marotta et al., 1998), and the systemic differences it causes are usually small or even inconsistent. The other method uses a visual illusion that induces the Roelofs effect, and this method causes large and consistent system differences. In this visual illusion, an object is surrounded by a frame positioned on one side of the observer's midline. The actor must verbally describe the location of the target in the presence of an offset frame and point to the target as soon as it disappears from the view. The results showed that even if participants had perceptual localization errors, the motion behavior toward the object was still accurate (Bridgeman, 1992; Bridgeman et al., 1997). 
This study aims to investigate the connection between pitch and motion speed, so the perception of motion speed is key to the study. In a target collision experiment, the human brain must estimate the target collision time accurately to intercept a moving object successfully. Since the speed of a moving object is key to predicting the collision time of a target (Assmus et al., 2005; Assmus et al., 2003; Bares et al., 2007, 2010; Beudel et al., 2009; Bosco et al., 2008; Coull et al., 2008; Field & Wann, 2005; Reilly & Mesulam, 2008), target collision experiments can be used to study the human perception of motion speed. In our target collision experiment, a small ball moved down the computer screen, and participants had to attempt to block the ball with a baffle. When the participant pressed the mouse, the baffle was released and moved directly to the right. Participants were tasked with releasing the baffle at the exact moment it was needed to block the ball. Besides the cognitive measures, our study obtained an action measure (Witt, 2018). Specifically, the time to release the baffle provided an action measure of perceived speed: If the ball looked faster, the participants should have released the baffle earlier. If the ball looked slower, the participants should have released the baffle later. 
Experiment 1 consisted of practice blocks and experimental blocks. Participants first took a practice block, all of which were carried out without tone, and no data were collected. Then they took an experimental block, all of which conducted with different tones. After each trial, participants were asked to rate the speed of the ball on a scale of 1–5. The speed score and baffle release time were recorded and used for analysis of variance. Participants were asked to rate the ball speed after each trial, which meant they had to make clear speed judgments. However, previous studies have shown that making explicit judgments may affect participants' perceived effects (Bridgeman, 1992; Vishton et al., 2007), so we designed Experiment 2. In this experiment, participants did not take a practice block or have to judge its speed after each trial. The final results showed that both explicit judgment and action measures showed the influence of the pitch-induced cuing effect on the motion measurement of perceived speed, and the pitch-induced cuing effect still pertained in the absence of explicit judgment. 
Experiment 1
Method
Participants
Twelve participants from the Air Force Engineering University (aged between 20 and 25 years, mean age: 23 years, SD = 1.04; six men and six women) took part in the Experiment 1 with informed consent. All of the participants reported having normal or corrected vision and hearing and were not informed of the purpose of the study. 
Stimulus and apparatus
The stimuli were displayed on a 51-cm × 29-cm monitor with 1,920 × 1,080 resolution. The participants sat 57 cm from the monitor screen, their head position stabilized with the help of a chin brace. The screen displayed a baffle and a small ball. The baffle on the screen was 6.0 cm long and 1.0 cm wide, and the center of the baffle was 22.0 cm from the left of the screen and 1.5 cm from the bottom of the screen. The ball had a radius of 1.0 cm, and its center was 27.5 cm from the left of the screen, and its vertical position was set to either 3.0 or 7.0 cm from the top of the screen. The ball moved at four speeds, which were 2.0, 4.0, 6.0, and 8.0 cm/s. The baffle moved at a fixed speed of 2 cm/s. Figure 1 shows the beginning of the trial with a low ball. 
Figure 1.
 
Target collision experiment scene.
Figure 1.
 
Target collision experiment scene.
Figure 2.
 
Relationship between average baffle release time and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 2.
 
Relationship between average baffle release time and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 3.
 
Relationship between average baffle release time and ball position and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 3.
 
Relationship between average baffle release time and ball position and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 4.
 
Relationship between average baffle release time and ball speed and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 4.
 
Relationship between average baffle release time and ball speed and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
At the beginning of each trial, the ball on the screen immediately moved down. Participants held down the left mouse button to release the baffle, which moved directly to the right at a constant speed. During the movement of the ball and the baffle, if the upper edge of the baffle blocked the path of the ball, the ball stopped on the baffle, and the trial was deemed successful. Otherwise, the ball would not land on the top edge of the baffle, and the trial was considered a failure to block. 
Experimental procedure
Participants tried to block a ball with a baffle. When the participant pressed the mouse, the baffle was released and moved directly to the right. The speed and height of the ball were the variables in the experiment. 
The whole experiment consisted of practice blocks and experimental blocks. Each block contained 24 trials with all combinations of pitch (high, low, and none), ball speed (2.0, 4.0, 6.0, and 8.0 cm/s), and ball position (high and low). The order was counterbalanced across participants. After each trial, a screen with the word “next” was presented for 4,000 ms before the next trial began. Participants first took a practice block, five times in each condition, for a total of 120 trials. All of trials were carried out without tone, and no data were collected. Then they took an experimental block, 10 times in each condition, for a total of 240 trials. After each trial, participants were asked to rate the speed of the ball on a scale of 1–5 but received no feedback on speed judgments. The speed score and baffle release time were recorded and used for analysis of variance. Before the experiment began, participants were informed that auditory stimulation provided no information about the ball's speed. 
In the experimental block, the stimuli used in some trials came in pairs, one auditory and one visual, with the same beginning. At the beginning of these trials, the ball on the screen immediately moved down, and a high (1,500 Hz) or low (300 Hz) sinusoidal tone that lasted for 10 s started playing immediately. Sound was presented at a comfortable listening level and played through speakers on the left and right sides of the computer screen. Other trials were carried out without tone as a control group. 
Results
The data collected in the experiment were divided into two parts: One was the baffle release time, which was the reaction time of the participants from the beginning of the trial to the pressing of the left mouse button. The other was the score of the participants on the speed of the ball, and the higher the score, the faster the ball was thought to be moving by the participants. These data were processed by R software, and the afex package was used to analyze the significant differences between different experimental factors and their interactions. 
The experimental data of the response time of the participants were used for repeated-measures analysis of variance (ANOVA) with pitch, ball speed, and ball position as the within-subjects factors. The results showed that the ball position significantly affected the baffle release time, F(1, 11) = 5,167.17, p < 0.001, ηp2 = 0.998. When the ball was lower, the participants released the baffle earlier. The ball speed significantly affected the baffle release time, F(2.36, 25.96) = 71,729.59, p < 0.001, ηp2 > 0.999. When the ball was faster, the participants released the baffle earlier. The interaction between ball speed and ball position was significant, F(1.91, 20.99) = 1,104.87, p < 0.001, ηp2 = 0.990. We focused on the effect of pitch on baffle release time. The results showed that baffle release time was significantly affected by pitch, F(1.35, 14.84) = 46.12, p < 0.001, ηp2 = 0.807 (see Figure 2). Additionally, the interaction between pitch and ball position was not significant, F(1.94, 21.39) = 0.01, p = 0.991, ηp2 < 0.001. The interaction between pitch and ball speed was significant, F(4.52, 49.72) = 4.27, p = 0.003, ηp2 = 0.280 (see Figures 3 and 4). The interaction of pitch, ball position, and ball speed was not significant, F(4.12, 45.36) = 0.15, p = 0.964, ηp2 = 0.014. 
The interaction between pitch and speed was significant, so the pairwise comparison test of pitch (high, low, none) was carried out, and the results are shown in Table 1. We found that when the ball was moving at a lower speed, there was no significant difference in response time between the high tone and the no tone; there was a significant difference in response time between high tone and low tone; the difference of reaction time between low tone and no tone was significant. On the contrary, when the ball was moving at a higher speed, there was no significant difference in response time between low tone and no tone; there was a significant difference in response time between high tone and low tone; there was a significant difference in response time between high tone and no tone. When the tone was lower, the participants released the baffle later to block the ball at a lower speed. When the pitch was higher, the participants released the baffle earlier to block the high-speed ball. What this means is that the low-speed ball was thought to move more slowly at a low tone, and the high-speed ball was thought to move faster at a high tone. 
Table 1.
 
Results of pairwise comparisons in Experiment 1.
Table 1.
 
Results of pairwise comparisons in Experiment 1.
After each blocking trial, participants rated the ball's speed. Match the speed score to the ball speed: The score 1 matched the ball speed of 2.0 cm/s, the score 2 matched the ball speed of 3.5 cm/s, the score 3 matched the ball speed of 5.0 cm/s, the score 4 matched the ball speed of 6.5 cm/s, and the score 5 matched the ball speed of 8.0 cm/s. These velocities were treated as perceived velocities and used to calculate the point of subjective equality (PSE) of ball speed, and a high PSE score meant the ball was considered to be moving faster. PSEs were used for repeated-measures ANOVA with pitch and ball position as within-subjects factors, as shown in Figure 5. The results showed that pitch significantly affected PSEs, F(1.87, 20.56) = 311.24, p < 0.001, ηp2 = 0.966. The influence of ball position on PSE was not significant, F(1, 11) = 0.25, p = 0.626, ηp2 = 0.022. The interaction between pitch and ball position was not significant, F(1.22, 13.42) = 0.32, p = 0.624, ηp2 = 0.028. 
Figure 5.
 
Relationship between average PSEs and ball position and pitch. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 5.
 
Relationship between average PSEs and ball position and pitch. The error bar is a 95% confidence interval calculated internally by the participants.
By calculating the effects of each measure, the effect of pitch on perceived speed can be compared directly. The effect is calculated by subtracting the score of the higher pitch from the score of the lower pitch (PSE or RT) and then dividing by the score of the higher pitch. The paired-sample t test showed that there was a significant difference between the two measures of the effect, t(11) = −8.47, p < 0.001. Figure 6 shows a comparison of the two measures. The higher the score, the more obvious the influence of pitch on the perceived speed of the ball. A positive score means that the higher the pitch, the faster the ball appears. Each point represents an individual participant, and each participant has a point for each of the two measures. The results show that both measurements show the pitch-induced cuing effect on the perceived speed of the ball. 
Figure 6.
 
Comparison of the effects of the two measures.
Figure 6.
 
Comparison of the effects of the two measures.
Figure 7.
 
Relationship between average baffle release time and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 7.
 
Relationship between average baffle release time and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 8.
 
Relationship between average baffle release time and ball position and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 8.
 
Relationship between average baffle release time and ball position and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 9.
 
Relationship between average baffle release time and ball speed and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 9.
 
Relationship between average baffle release time and ball speed and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Experiment 2
Method
Research has shown that making explicit judgments may affect participants' perceived effects. Therefore, an additional experiment must be designed. In Experiment 2, the experimental environment was the same as in Experiment 1. The difference was that participants did not take a practice block or have to judge the speed of the ball after each trial. 
The stimuli used in some trials came in pairs, one auditory and one visual, with the same beginning. At the beginning of these trials, the ball on the screen immediately moved down and a high (1,500 Hz) or low (300 Hz) sinusoidal tone that lasted for 10 s started playing immediately. Sound was presented at a comfortable listening level and played through speakers on the left and right sides of the computer screen. Other trials were carried out without tone as a control group. 
Twelve participants participated in Experiment 2; none of participants had participated in Experiment 1. Before the experiment began, participants had to move the baffle by clicking the left mouse button 10 times. They took an experimental block, 10 times in each condition, for a total of 240 trials. The baffle release time was recorded and used for analysis of variance. Before the experiment began, participants were informed that auditory stimulation provided no information about the ball's speed. 
Results
The experimental data of the response time of the participants were used for repeated-measures ANOVA with pitch, ball speed, and ball position as the within-subjects factors. The results showed that the ball position significantly affected baffle release time, F(1, 11) = 12,566.72, p < 0.001, ηp2 = 0.999. When the ball was lower, the participants released the baffle earlier. Baffle release time was significantly affected by ball speed, F(2.39, 26.29) = 61,266.12, p < 0.001, ηp2 > 0.999. When the ball was moving faster, the participants released the baffle earlier. The interaction between ball speed and ball position was significant, F(2.31, 25.42) = 2,297.45, p < 0.001, ηp2 = 0.995. Importantly, pitch significantly affected baffle release time, F(1.98, 21.83) = 40.63, p < 0.001, ηp2 = 0.787 (see Figure 7). Additionally, the interaction between pitch and ball position was not significant, F(1.80, 19.81) = 0.02, p = 0.968, ηp2 = 0.002. The interaction between pitch and ball speed was significant, F(3.70, 40.65) = 3.58, p = 0.016, ηp2 = 0.246 (see Figures 8 and 9). The interaction of pitch, ball position, and speed was not significant, F(2.62, 28.87) = 0.32, p = 0.783, ηp2 = 0.029. The results showed that the pitch still affected baffle release time even when a clear judgment was not made. 
The interaction between pitch and speed was significant, so the pairwise comparison test of pitch (high, low, none) was carried out, and the results are shown in Table 2. The results obtained are similar to those obtained in Experiment 1: The low-speed ball was thought to move more slowly at a low tone, and the high-speed ball was thought to move faster at a high tone. 
Table 2.
 
Results of pairwise comparisons in Experiment 2.
Table 2.
 
Results of pairwise comparisons in Experiment 2.
Analysis by synthesis
Data from the two experiments were combined to determine whether making an explicit judgment affected the size of the effect of pitch on the baffle release time. With the experiment as the intersubject factor and pitch, ball speed, and ball position as the within-subjects factors, repeated-measures ANOVA was performed. The ANOVA results of all factors and their interactions are shown in Table 3
Table 3.
 
ANOVA results of all factors and their interactions.
Table 3.
 
ANOVA results of all factors and their interactions.
The results showed that baffle release time was significantly affected by pitch, F(1.75, 38.41) = 86.50, p < 0.001, ηp2 = 0.797. Furthermore, the interaction between the experiment and pitch was not significant, F(1.75, 38.41) = 0.07, p = 0.912, ηp2 = 0.003 (Figure 10). Therefore, when an explicit judgment was not made, pitch still influenced the action measure of baffle release time, and the speed judgment did not influence the effect of pitch on the baffle release time. 
Figure 10.
 
In both experiments, the mean baffle release time per participant at the lower pitch was subtracted from the mean baffle release time at the higher pitch. A positive difference score indicated that participants released the baffle earlier at a higher pitch than at a lower pitch. The larger the difference, the more significant the effect of pitch on baffle release time.
Figure 10.
 
In both experiments, the mean baffle release time per participant at the lower pitch was subtracted from the mean baffle release time at the higher pitch. A positive difference score indicated that participants released the baffle earlier at a higher pitch than at a lower pitch. The larger the difference, the more significant the effect of pitch on baffle release time.
Discussion
The purpose of this experiment is to explore the relationship between pitch and motion speed. According to the data, even though the ball was moving at the same speed, there was a significant difference in the release time it took for the participants to release the baffle at different tones. In this experiment, the motion measurement of releasing the baffle could be used to evaluate the perceived speed of the ball. Specifically, when the participants thought the ball was moving faster, the earlier the movement of releasing the baffle occurred; on the contrary, when the participants thought the ball was moving slower, the later they released the baffle. 
In our experiment, the length and moving speed of the baffle as well as the falling height and speed of the ball determined the success rate of the baffle blocking the ball. Most of the participants successfully blocked falling balls during the trials. This design for conducting experiments without tone provides a reference for experimental data analysis. Compared with the release time without tone, when the participants released the baffle for a longer time, the moving speed of the ball was underestimated. On the contrary, when the participants released the baffle for a shorter time, the speed of the ball was overestimated. 
In Experiment 1, we used two measures: One was the baffle release time, that is, the reaction time of the participants from the beginning of the experiment to the press of the left mouse button, which could reflect the action effect of the participants. The other was the rating of the ball speed: The higher the score was, the faster the ball was thought to be, which could reflect the participants' perception effect. In the analysis of the baffle release time, we found that the pitch significantly affected the baffle release time, while the interaction between pitch and height was not significant, and the interaction between pitch and speed was significant. In addition, under the influence of low tone, the time of releasing the baffle to block the low-speed ball was significantly longer, which indicated that the participants generally believed that the low-speed ball moved more slowly under the influence of low tone. On the contrary, under the influence of high tone, the time of releasing the baffle to block the high-speed ball was significantly shorter, which indicated that the participants generally believed that the high-speed ball moved faster under the influence of high tone. In addition, we found that the effect of high tone on the speed perception of low-speed balls and low tone on the speed perception of high-speed balls was not significant when compared with the release time without tone. PSE analysis more intuitively reflects the perceived effect of pitch on speed: Pitch significantly affects perceived speed. Compared with the speed score without pitch, the speed score at the low tone was significantly lower, and the speed score at the high tone was significantly higher. The experimental data of PSE and baffle release time both show that the low-speed ball appeared slower under the influence of low tone, while the high-speed ball appeared faster under the influence of high tone. 
In the Experiment 2, participants did not know how fast the ball was moving before the experiment and were not asked to rate the speed of the ball after each trial. From the data analysis, we still found the effect of pitch on the baffle release time, and the interaction between pitch and height was not significant, while the interaction between pitch and speed was significant. This is consistent with the result of Experiment 1. Similarly, under the influence of low tone, the time of releasing the baffle to block the low-speed ball was significantly longer. On the contrary, under the influence of high tone, the time of releasing the baffle to block the high-speed ball was significantly shorter. Combining the data from Experiment 1 and Experiment 2, we found that the effect of pitch on the baffle release time was significant, while the interaction between experiment and pitch was not significant. Therefore, we concluded that pitch significantly affects baffle release time regardless of whether an explicit speed judgment was made, suggesting that the association between pitch and motion speed was not arbitrary and that there is indeed a consistent mapping. 
There is a consistent mapping between pitch and vertical position, with high pitch and low pitch causing attention to shift to higher or lower positions. This relationship may affect participants' judgments of the ball's falling height, which in turn affects the baffle release time. However, our results ruled out this possibility: The interaction between pitch and height was not significant in the analysis of the baffle release time data from Experiment 1 and Experiment 2, as well as the PSE data from Experiment 1, indicating that different pitch effects did not affect participants' judgments of the ball height. However, the interaction between pitch and speed is significant. Specifically, under the influence of low tone, the moving speed of the low-speed ball was underestimated; under the influence of high tone, the speed of the high-speed ball was overestimated. Thus, the effect of pitch on the baffle release time depended on participants' judgments of the ball's speed of movement. 
The release of the baffle was catapulted, so after the button was clicked, the participant was unable to influence the experiment in any way. This was for the initial action and did not involve successive actions. Therefore, both the explicit judgment and the release time of the baffle can reflect the effect of pitch, because both measures are driven by the same perceptual information. Moreover, this experiment excluded explanations related to memory. In some experiments, participants were required to estimate the properties of certain objects without exposure, thus concluding that it was memory rather than perception that affects perception (Witt & Proffitt, 2005). However, in our experiment, the movement of the ball was continuously visible, and the movement of releasing the baffle occurred during the movement of the ball, so memory was not involved in the whole experiment. In addition, in our Experiment 1, the participants would take an experimental block before the experimental block, which weakened the training and learning ability of the participants in the process to some extent. From the above analysis, it appeared that auditory pitch did affect the perception of motion speed. The low-speed ball appeared slower under the influence of low tone, while the high-speed ball appeared faster under the influence of high tone. 
Conclusion
There is a cross-modal mapping relationship between auditory pitch and several visual properties, such as size, shape, light and shade, and vertical position, but the relationship between auditory pitch and motion speed is unexplored. This study focused on the correlation between auditory pitch and motion speed. Besides the clear judgment of the speed of motion, this experiment also used an action measure, that is, releasing a baffle to block the movement of the ball. The data showed that regardless of whether there was a clear judgment of motion speed, the timing to release the baffle was influenced by auditory pitch. Since the baffle was released as the ball moved and there was no additional blocking strategy, the results ruled out explanations related to memory and need. The experiment showed that auditory pitch affects the perception of motion speed, with the high-speed ball appearing faster at high pitch and the low-speed ball appearing slower at low pitch. 
Acknowledgments
Supported by research on an adaptive human–machine interface design method to multiseat multitask control cabin (51675530) and research on the intuitive interaction design method to fast decision-making command and control system interface under the pressure of time and information (52175282). 
Commercial relationships: none. 
Corresponding author: Wei Wang. 
Address: Air and Missile Defense College, Air Force Engineering University, Xi'an, China. 
References
Assmus, A., Marshall, J. C., Noth, J., Zilles, K., & Fink, G. R. (2005). Difficulty of perceptual spatiotemporal integration modulates the neural activity of left inferior parietal cortex. Neuroscience, 132(4), 923–927, doi:10.1016/j.neuroscience.2005.01.047. [CrossRef]
Assmus, A., Marshall, J. C., Ritzl, A., Noth, J., Zilles, K., & Fink, G. R. (2003). Left inferior parietal cortex integrates time and space during collision judgments. NeuroImage, 20(Suppl. 1), S82–S88, doi:10.1016/j.neuroimage.2003.09.025.
Bares, M., Lungu, O., Tao, L., et al. (2007). Bares M, Lungu O, Liu T, Waechter T, Gomez CM, Ashe J. Impaired predictive motor timing in patients with cerebellar disorders. Experimental Brain Research, 180(2), 355–365, doi:10.1007/s00221-007-0857-8. [CrossRef]
Bares, M., Lungu, O. V., Husarova, I., & Gescheidt, T. (2010). Predictive motor timing performance dissociates between early diseases of the cerebellum and Parkinson's disease. Cerebellum, 9(1), 124–135, doi:10.1007/s12311-009-0133-5. [CrossRef]
Belkin, K., Martin, R., Kemp, S. E., & Gilbert, A. N. (1997). Auditory pitch as a perceptual analogue to odor quality. Psychological Science, 8(4), 340–342, doi:10.1111/j.1467-9280.1997.tb00450.x [CrossRef]
Beudel, M., Renken, R., Leenders, K. L., & Jong, B. M. D. (2009). Cerebral representations of space and time. NeuroImage, 44(3), 1032–1040, doi:10.1016/j.neuroimage.2008.09.028.
Bosco, G., Carrozzo, M., & Lacquaniti, F. (2008). Contributions of the human temporoparietal junction and MT/V5þ to the timing of interception revealed by transcranial magnetic stimulation. Journal of Neuroscience, 28(46), 12071–12084, doi:10.1523/JNEUROSCI.2869-08.2008.
Brenner, E., & Smeets, J. B. (1996). Size illusion influences how we lift but not how we grasp an object. Experimental Brain Research, 111(3), 473–476, doi:10.1007/bf00228737.
Bridgeman, B. (1992). Conscious vs unconscious processes: The case of vision. Theory & Psychology, 2(1), 73–88, doi:10.1177/0959354392021004.
Bridgeman, B., Peery, S., & Anand, S. (1997). Interaction of cognitive and sensorimotor maps of visual space. Perception & Psychophysics, 59(3), 456–469, doi:10.3758/BF03211912.
Brochard, R., Dufour, A., & Despres, O. (2004). Effect of musical expertise on visuospatial abilities: Evidence from reaction times and mental imagery. Brain & Cognition, 54(2), 103–109, doi:10.1016/S0278-2626(03)00264-1.
Bruzzi, E., Talamini, F., Priftis, K., & Grassi, M. (2017). A SMARC effect for loudness. i-Perception, 8(6), 204166951774217, doi:10.1177/2041669517742175.
Cheek, J. M., & Smith, L. R. (1999). Music training and mathematics achievement. Adolescence, 34(136), 759–761, doi:10.1177/105971239900700311.
Coull, J. T., Vidal, F., Goulon, C., Nazarian, B., & Craig, C. (2008). Using time-to-contact information to assess potential collision modulates both visual and temporal prediction networks. Frontiers in Human Neuroscience, 2, 10, doi:10.3389/neuro.09.010.2008.
Spence, C., & Driver, J. (Eds.). (2004). Crossmodal space and crossmodal attention. Oxford, UK: Oxford University Press.
Daprati, E., & Gentilucci, M. (1997). Grasping an illusion. Neuropsychologia, 35(12), 1577–1582, doi:10.1016/S0028-3932(97)00061-4.
Evans, K. K., & Treisman, A. (2010). Natural cross-modal mappings between visual and auditory features. Journal of Vision, 10(1), 1–12, doi:10.1167/10.1.6.
Field, D. T., & Wann, J. P. (2005). Perceiving time to collision activates the sensorimotor cortex. Current Biology, 15(5), 453–458, doi:10.1016/j.cub.2004.12.081.
Frens, M. A., Van Opstal, A. J., & Van der Willigen, R. F. (1995). Spatial and temporal factors determine audio-visual interactions in human saccadic eye movements. Perception & Psychophysics, 57(6), 802–816, doi:10.3758/bf03206796.
Calvert, G. A., Spence, C., & Stein, B. E. (Eds.). (2004). The handbook of multisensory processes. Cambridge, MA: MIT Press.
Gal, D., Wheeler, S. C., & Shiv, B. (2007). Cross-modal influences on gustatory perception. Manuscript submitted for publication. Retrieved from http://ssrn.com/abstract=1030197.
Gallace, A., & Spence, C. (2006). Multisensory synesthetic inter-actions in the speeded classification of visual size. Perception & Psychophysics, 68(7), 1191–1203, doi:10.3758/BF03193720.
Gilbert, A. N., Martin, R., & Kemp, S. E. (1996). Cross-modal correspondence between vision and olfaction: The color of smells. American Journal of Psychology, 109(3), 335–351, doi:10.2307/1423010.
Grassi, M. (2005). Do we hear size or sound? Balls dropped on plates. Perception & Psychophysics, 67(5), 274–284, doi:10.3758/BF03206491.
Grassi, M., Pastore, M., & Lemaitre, G. (2013). Looking at the world with your ears: How do we get the size of an object from its sound? Acta Psychologica (Amst), 143(1), 96–104, doi:10.1016/j.actpsy.2013.02.005.
Haffenden, A. M., & Goodale, M. A. (1998). The effect of pictorial illusion on prehension and perception. Journal of Cognitive Neuroscience, 10(1), 122–136, doi:10.1162/089892998563824.
Jones, J. A., & Jarick, M. (2006). Multisensory integration of speech signals: The relationship between space and time. Experimental Brain Research, 174(3), 588–594, doi:10.1007/s00221-006-0634-0.
Kemp, S. E., & Gilbert, A. N. (1997). Odor intensity and color lightness are correlated sensory dimensions. American Journal of Psychology, 110(1), 35–46, doi:10.2307/1423699.
Keus, I. M., Jenk, K., & Schwarz, W. (2005). Psychophysiological evidence that the SNARC effect has its functional locus in a response selection stage. Brain Research: Cognitive Brain Research, 24(1), 48–56, doi:10.1016/j.cogbrainres.2004.12.005.
Lidji, P., Kolinsky, R., Lochy, A., & Morais, J. (2007). Spatial associations for musical stimuli: A piano in the head? Journal of Experimental Psychology: Human Perception and Performance, 33(5), 1189–1207, doi:10.1037/0096-1523.33.5.1189.
Maniglia, M., Grassi, M., & Ward, J. (2017). Sounds are perceived as louder when accompanied by visual movement. Multisensory Research, 30(2), 159–177, doi:10.1163/22134808-00002569.
Marks, L. E. (1987). On cross-modal similarity: Auditory–visual interactions in speeded discrimination. Journal of Experimental Psychology: Human Perception and Performance, 13(3), 384–394, doi:10.1037/0096-1523.13.3.384.
Marotta, J. J., DeSouza, J. F., Haffenden, A. M., & Goodale, M. A. (1998). Does a monocularly presented size-contrast illusion influence grip aperture? Neuropsychologia, 36(6), 491–497, doi:10.1016/S0028-3932(97)00154-1.
Martino, G., & Marks, L. E. (1999). Perceptual and linguistic interactions in speeded classification: Tests of the semantic coding hypothesis. Perception, 28(7), 903–923, doi:10.1068/p2866.
Melara, R. D. (1989). Dimensional interaction between color and pitch. Journal of Experimental Psychology: Human Perception and Performance, 15(1), 69–79, doi:10.1037/0096-1523.15.1.69.
Melara, R. D., & Marks, L. E. (1990). Processes underlying dimensional interactions: Correspondences between linguistic and nonlinguistic dimensions. Memory & Cognition, 18(5), 477–495, doi:10.3758/BF03198481.
Moore, B. C. J. (1977). An introduction to the psychology of hearing. Archives of Otolaryngology, 103(12): 745–746, doi:10.1001/archotol.1977.00780290081022.
Nava, E., Grassi, M., & Turati, C. (2016). Audio-visual, visuo-tactile and audio-tactile correspondences in preschoolers. Multisensory Research, 29(1–3), 93–111, doi:10.1163/22134808-00002493.
Patching, G. R., & Quinlan, P. T. (2002). Garner and congruence effects in the speeded classification of bimodal signals. Journal of Experimental Psychology: Human Perception and Performance, 28(4), 755–775, doi:10.1037//0096-1523.28.4.755.
Pitteri, M., Marchetti, M., Grassi, M., & Priftis, K. (2021). Pitch height and brightness both contribute to elicit the SMARC effect: A replication study with expert musicians. Psychological Research, 85(6), 2213–2222, doi:10.1007/s00426-020-01395-0.
Pitteri, M., Marchetti, M., Priftis, K., & Grassi, M. (2017). Naturally together: pitch-height and brightness as coupled factors for eliciting the SMARC effect in non-musicians. Psychological Research, 81(1), 243–254, doi:10.1007/s00426-015-0713-6
Reilly, O., & Mesulam, J. X. (2008). The cerebellum predicts the timing of perceptual events. Journal of Neuroscience, 28(9), 2252–2260, doi:10.1523/JNEUROSCI.2742-07.2008.
Rusconi, E., Kwan, B., Giordano, B. L., Umilta, C., & Butterworth, B. (2006). Spatial representation of pitch height: The SMARC effect. Cognition, 99(2), 113–129, doi:10.1016/j.cognition.2005.01.004.
Sandrini, M., Rossini, P., & Miniussi, C. (2004). The differential involvement of inferior parietal lobule in number comparison: A rTMS study. Neuropsychologia, 42(14), 1902–1909, doi:10.1016/j.neuropsychologia.2004.05.005.
Schmithorst, V. J., & Holland, S. K. (2003). The effect of musical training on music processing: A functional magnetic resonance imaging study in humans. Neuroscience Letters, 348(2), 65–68, doi:10.1016/S0304-3940(03)00714-6.
Shore, D. I., Barnes, M. E., & Spence, C. (2006). Temporal aspects of the visuotactile congruency effect. Neuroscience Letters, 392(1–2), 96–100, doi:10.1016/j.neulet.2005.09.001.
Slutsky, D. A., & Recanzone, G. H. (2001). Temporal and spatial dependency of the ventriloquism effect. NeuroReport, 12(1), 7–10, doi:10.1097/00001756-200101220-00009.
Spence, C. (2011). Crossmodal correspondences: A tutorial review. Attention, Perception & Psychophysics, 73(4), 971–995, doi:10.3758/s13414-010-0073-7.
van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45(3), 598–607, doi:10.1016/j.neuropsychologia.2006.01.001.
Vishton, P. M., Sephens, N. J., Nelson, L. A., Morra, S. E., Brunick, K. L., & Stevens, J. A. (2007). Planning to reach for an object changes how the reacher perceives it. Psychological Science, 18(8), 713–719, doi:10.1111/j.1467-9280.2007.01965.x.
Witt, J. K. (2018). In absence of an explicit judgment, action-specific effects still influence an action measure of perceived speed. Conousness & Cognition, 64, 95–105, doi:10.1016/j.concog.2018.04.017.
Witt, J. K., & Proffitt, D. R. (2005). See the ball, hit the ball–Apparent ball size is correlated with batting average. Psychological Science, 16(12), 937–938, doi:10.1111/j.1467-9280.2005.01640.x.
Figure 1.
 
Target collision experiment scene.
Figure 1.
 
Target collision experiment scene.
Figure 2.
 
Relationship between average baffle release time and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 2.
 
Relationship between average baffle release time and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 3.
 
Relationship between average baffle release time and ball position and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 3.
 
Relationship between average baffle release time and ball position and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 4.
 
Relationship between average baffle release time and ball speed and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 4.
 
Relationship between average baffle release time and ball speed and pitch in Experiment 1. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 5.
 
Relationship between average PSEs and ball position and pitch. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 5.
 
Relationship between average PSEs and ball position and pitch. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 6.
 
Comparison of the effects of the two measures.
Figure 6.
 
Comparison of the effects of the two measures.
Figure 7.
 
Relationship between average baffle release time and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 7.
 
Relationship between average baffle release time and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 8.
 
Relationship between average baffle release time and ball position and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 8.
 
Relationship between average baffle release time and ball position and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 9.
 
Relationship between average baffle release time and ball speed and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 9.
 
Relationship between average baffle release time and ball speed and pitch in Experiment 2. The error bar is a 95% confidence interval calculated internally by the participants.
Figure 10.
 
In both experiments, the mean baffle release time per participant at the lower pitch was subtracted from the mean baffle release time at the higher pitch. A positive difference score indicated that participants released the baffle earlier at a higher pitch than at a lower pitch. The larger the difference, the more significant the effect of pitch on baffle release time.
Figure 10.
 
In both experiments, the mean baffle release time per participant at the lower pitch was subtracted from the mean baffle release time at the higher pitch. A positive difference score indicated that participants released the baffle earlier at a higher pitch than at a lower pitch. The larger the difference, the more significant the effect of pitch on baffle release time.
Table 1.
 
Results of pairwise comparisons in Experiment 1.
Table 1.
 
Results of pairwise comparisons in Experiment 1.
Table 2.
 
Results of pairwise comparisons in Experiment 2.
Table 2.
 
Results of pairwise comparisons in Experiment 2.
Table 3.
 
ANOVA results of all factors and their interactions.
Table 3.
 
ANOVA results of all factors and their interactions.
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×