Perceptual decision processes flexibly adapt to avoid change-of-mind motor costs

Abstract

Abstract

Abstract:

Abstract  The motor system is tightly linked with perception and cognition. Recent studies have shown that even anticipated biophysical action costs associated with competing response options can be incorporated into decision-making processes. As a result, choices associated with high energy costs are less likely to be selected. However, some action costs may be harder to predict. For example, a person choosing among apples at a grocery store may change his or her mind suddenly about which apple to put into the cart. This change of mind may be reflected in motor output as the initial decision triggers a motor response toward a Granny Smith that is subsequently redirected toward a Red Delicious. In the present study, to examine how motor costs associated with changes of mind affect perceptual decision making, participants performed a difficult random dot–motion discrimination task in which they had to indicate the direction of motion by reaching to one of two response options. Although each response box was always equidistant from the starting position, the physical distance between the two response options was varied. We found that when the boxes were far apart from one another, and thus changes of mind incurred greater redirection motor costs, change-of-mind frequency decreased while latency to initiate movement increased. This occurred even when response box distance varied randomly from trial to trial and was cued only 1 s before each trial began. Thus, we demonstrated that observers can dynamically adjust perceptual decision-making processes to avoid high motor costs incurred by a change of mind.

Introduction

Classical theories of cognitive psychology have often assumed that cognition operates in discrete stages (e.g., Donders, 1969; Fodor, 1983; Padoa-Schioppa, 2011; Sternberg, 1969). According to this view, the motor system merely reflects the output of higher-level cognitive processes, such as perception and decision making. However, recent studies have cast doubt on this idea by demonstrating strong connections, both behaviorally and neurophysiologically, between the motor system, perception, and cognition (e.g., Cisek & Kalaska, 2005, 2010; Gallivan, McLean, Smith, & Culham, 2011; Song & Nakayama, 2009; Song & Nakayama, 2006; Song, Rafal, & McPeek, 2011; Spivey, Grosjean, & Knoblich, 2005). In particular, action output is often initiated before decision-making processes have finalized; thus, movement trajectories may reflect otherwise internal decision-making dynamics (e.g., Albantakis & Deco, 2011; McPeek & Keller, 2001; Resulaj, Kiani, Wolpert, & Shadlen, 2009; Song & Nakayama, 2009; Song & Nakayama, 2006; Spivey et al., 2005). 

In addition to reflecting perceptual and cognitive processes, movement output can also influence perceptual and cognitive processes. For example, in a recent study, we found that previous motor responses affect target selection for action (Moher & Song, 2013). In that study, observers occasionally changed their mind after initially triggering a motor plan toward one response option, resulting in a “partial error” response (similar to Song & Nakayama, 2008). We found that when task context was kept consistent via target color repetition, trials following partial errors were likely to exhibit less efficient movements with more pull of hand trajectories toward nontarget objects. A number of other studies have also shown that previous reach movements can affect goal-directed action (e.g., Griffiths & Tipper, 2009; Jax & Rosenbaum, 2007, 2009). 

Even anticipated biomechanical costs associated with different movement outcomes contribute to decision-making processes. Cos, Belanger, and Cisek (2011) found that when one possible motor response carried a higher energy cost than an alternative motor response, participants were biased to select the less costly response. Thus, not only are movements generally optimized in order to minimize distance traveled (Morasso, 1981) and physical energy costs (Sabes & Jordan, 1997), but decisions themselves are biased toward more efficient movement options when multiple action options compete for selection. 

A distributed consensus framework (Cisek, 2012) has been suggested for understanding these interactions between perception, cognition, and action (see also, e.g., Song & Nakayama, 2009; Spivey et al., 2005). According to this framework, there is competition for decision outcomes at multiple levels of representation, among both higher-level goals and possible action outputs (including their associated costs). Decisions are executed when this competition is resolved. Thus, decision-making processes take into consideration not only high-level goals, but also the costs associated with various action outcomes. 

However, in some cases, it may be quite difficult to anticipate action costs. For example, reach movement trajectories to a target stimulus can vary a great deal from one trial to the next when there is competition for selection between a target and distractors (e.g., Moher & Song, 2013; Song & Nakayama, 2008; Song & Nakayama, 2006). This occurs because movement planning for alternative decisions in saccades and reaches can progress in parallel before movement initiation and continue even after a motor plan is initiated (e.g., Song & Nakayama, 2008; McPeek, Skavenski, & Nakayama, 2000). Thus, when observers change their mind about attention allocation, target selection, or a perceptual decision, movement trajectories can reflect this change of mind (see also, Albantakis & Deco, 2011; McPeek & Keller, 2001; Resulaj et al., 2009; Song & Nakayama, 2009; Song & Nakayama, 2006; Spivey et al., 2005). In these change-of-mind trials, the participant's hand traverses a longer path of movement and the movement takes longer to execute relative to trials in which no change of mind occurs. Thus, changes of mind are costly actions in terms of time (e.g., Song & Nakayama, 2008) and physical energy. 

In the present study, we examine whether these anticipated costs of a change of mind are incorporated into the decision-making process. This type of “metadecision” process—that is, a decision process that incorporates information about what will happen if the initial choice is overridden by a change of mind and adjusts accordingly—would suggest strong integration between perception, cognition, and action. 

We examined this question in a perceptual decision-making task by varying the distance between competing response options. We combined approaches from previous visually guided reaching studies (e.g., Moher & Song, 2013; Song & Nakayama, 2008) with a perceptual decision-making approach (e.g., Newsome, Britten, & Movshon, 1989; Resulaj et al., 2009). Participants had to indicate the direction of a random dot motion stimulus by reaching to one of two possible target boxes, one on either side of fixation. When the target boxes were placed close together in space, the cost of a change of mind was relatively low as redirection of the reach movement required a small change in the direction of movement trajectory (see Figure 1). Conversely, when the boxes were farther apart, changes of mind were quite costly, requiring a significant redirection of movement trajectory. If decision processes can be adjusted to account for the motor costs associated with changes of mind, we would expect changes of mind to occur less often when the competing response options are farther apart. 

Figure 1

 

Two sample low-coherence trials from Experiment 1. (A) Close-condition trial with high coherence, dots moving leftward. (B) Far-condition trial with low coherence, dots moving rightward. These trial types appeared in separate blocks in Experiment 1. The observer was required to indicate which direction the semicoherent cloud of dots was moving by pointing to the response box on the corresponding side of space. The distance from the starting position to the response boxes was equal across all conditions, but the distance between response boxes was greater in the far response box condition (B) compared to the close response box condition (A).

Experiment 1: Do decision processes adjust to avoid costly changes of mind?

In each trial, participants indicated which direction they thought a random dot motion stimulus was moving by reaching to a response box on that side of fixation. The distance between response boxes was varied by block. When response boxes were far apart from each other in space, we anticipated that change-of-mind frequency would be reduced, reflecting a change in the perceptual decision-making process to account for motor costs incurred by changes of mind. 

Materials and methods

Fourteen Brown University undergraduates and community members (nine male, mean age = 19.5 years) participated in a session lasting approximately 1 hr in exchange for either course credit or monetary compensation. All participants were right-handed with normal or corrected-to-normal color vision. The protocol was approved by the Brown University Institutional Review Board. All procedures reported in this and subsequent experiments followed the guidelines of the Declaration of Helsinki. 

Stimuli

All stimuli were displayed on a black background. In each trial, a white fixation cross measuring 0.7 by 0.7 cm appeared at the center of the screen. Response boxes with white borders (border width of 0.8 mm) measuring 2.8 cm² appeared on either side of fixation. In the close condition, the boxes appeared at an angular separation of 45° from each other from the starting position, measuring 2.1 cm above fixation on the vertical axis and 5.6 cm away from fixation on the horizontal axis, measured from the center of fixation to the center of the box. In the far condition, the boxes appeared an angular separation of 75° from the starting position, 0.4 cm below fixation and 11.4 cm away on the horizontal axis. Response boxes across all conditions were equidistant from the observer's starting finger position. 

In addition, 250 red dots appeared in each trial measuring 0.8 mm each within a centrally presented aperture with a diameter of 4.8 cm. In each trial, each dot remained on screen for 50 ms before being replotted. Dots were either moving to the left or right at a speed of 6.9 cm/s or were randomly replotted every 50 ms. Dot behavior was dependent on the dot coherence level of dots, meaning the percentage of dots that were moving in a single direction rather than being replotted randomly. The replotting of all dots was staggered such that approximately one third of all dots changed position during every screen refresh (approximately 16.66 ms). These methods were largely adapted from previous random dot motion tasks (e.g., Newsome et al., 1989; Resulaj et al., 2009; Roitman & Shadlen, 2002). 

Design and procedure

Each session began with a nine-point calibration procedure for reach movements. Following this, a brief practice block occurred, in which a staircase procedure was used to determine the optimal level of dot coherence for each subject. The staircase procedure was aimed at discovering a dot coherence level at which participants would perform at approximately 60% accuracy to optimize the frequency of changes of mind (e.g., Resulaj et al., 2009). 

We used an alternating staircase procedure (e.g., Cornsweet, 1962) to determine the dot coherence for each trial. One staircase began at 50% dot coherence (high-start staircase), and the other started at 2% coherence (low-start staircase). The staircases were alternated such that the first trial dot coherence was determined by the low-start staircase (i.e., 2%), the second by the high-start staircase (i.e., 50%), the third by the low-start staircase (i.e., dependent on whether the response on the previous low-start staircase trial was accurate), etc. For the low-start staircase, dot coherence was increased after two correct answers or decreased after one incorrect answer. Dot coherence in the high-start staircase increased following one correct answer or decreased following one incorrect answer. The magnitude of this increase or decrease varied depending on how many reversals of coherence had occurred. A reversal was defined as a change in dot coherence in the opposite direction of the previous change in dot coherence for that particular staircase (i.e., the previous shift in dot coherence was an increase for the low-start staircase, but due to two consecutive correct answers, the coherence was now shifted lower). 

For the first reversal of staircase direction, the coherence level moved up or down 10% for the high-start staircase and 8% for the low-start staircase (the latter being a lower jump because it took two consecutive correct answers to change and was therefore less volatile). Each subsequent reversal resulted in a smaller magnitude dot-coherence change with the magnitude being reduced 2% each time (i.e., the second reversal caused an 8% change for the high-start staircase and a 6% change for the low-start staircase). The magnitude stopped changing for both staircases once it reached 2%. The only exception is that the coherence never moved below 2% or above 50%. The staircase procedure finished when both staircases had reversed direction at least eight times. The coherence level of the two staircases was then averaged (and rounded up to the next even number if it was an odd number) to determine the coherence level to be used for the main experiment for that participant. 

Each trial during the staircase procedure began with a fixation cross that appeared at the center of the display as long as participants were within 2.54 cm of the starting position. Simultaneously, two response boxes appeared, one on either side of fixation. The response box distance (far vs. close; Figure 1) varied randomly from trial to trial. The fixation cross was removed to reveal the moving dots after 1 s. Participants were instructed to determine the direction of dot motion and touch the response box on the corresponding side of fixation (e.g., if the participant thought the dots were moving to the right side, they should reach to the response box on the right side of fixation). The dots disappeared as soon as observers moved at least 2.54 cm away from the starting position. Feedback was given in the form of a high beep for correct answers and a low beep for incorrect answers. There was a time limit to respond of 2.5 s, after which the trial was terminated and considered incorrect. Otherwise, the trial was terminated after participants made their response. A blank black screen appeared and remained on screen for 1 s after each trial. 

Following the staircase procedure, the main experiment began. The trial structure for the main experiment was similar to the staircase procedure. However, for the dot coherence level, a randomly selected 30% of all trials were set at 100% coherence (high coherence). These were included in order to obtain baseline movement trajectories and because inclusion of easier trials encourages rapid initiation of movement (e.g., Song & Nakayama, 2007), which is likely to increase the frequency of changes of mind. The remaining 70% of trials were set at the staircase level of coherence for each individual participant (low coherence; mean coherence level = 9.9%). The response box distance (far vs. close) was fixed within each block. An equal number of close and far blocks were presented for each participant and randomly ordered for each participant. The response boxes remained on screen during the intertrial interval. 

Participants completed 10 blocks of trials lasting 50 trials each. Participants were given an opportunity to rest between each block. Each session lasted approximately 1 hr. 

Data analysis

Data analysis was largely adapted from methods reported in Moher and Song (2013). When the participant's finger came within approximately 1.3 cm of the display on the z dimension and simultaneously within approximately 3.9 cm of the center of the response box on the x and y dimensions within the time limit, a response was considered correct. If this threshold was passed for a nontarget box or the participant did not cross any reaching threshold during the time limit, the trial was counted as incorrect. 

Hand-movement data were analyzed offline using custom MATLAB (Mathworks) software. Three-dimensional resultant speed scalars were created for each trial using a differentiation procedure in MATLAB. These scalars were then submitted to a second order, low-pass Butterworth filter with a cutoff of 10 Hz. Movement onset was calculated as the first time point in each trial after stimulus onset at which hand-movement speed exceeded 25.4 cm/s. Movement offset was defined as the first subsequent measurement in each trial when speed decreased to below 25.4 cm/s. Initiation latency was defined as the time elapsed between stimulus onset and movement onset. Movement time was defined as the time elapsed between movement onset and movement offset. 

To classify change-of-mind trials, we resampled each trial to create 101 samples equally spaced in time along the x dimension (e.g., Moher & Song, 2013; Song & Nakayama, 2008; Spivey et al., 2005). We then calculated the average movement toward each response box from all correct responses in the high-coherence condition and created a “zone” surrounding the average movement calculated as the distance 1.5 standard deviations in each direction from each time point along the x dimension (e.g., Moher & Song, 2013; Song & Nakayama, 2008). Changes of mind were defined as trials in which the participant's hand left the zone of the final response and entered the zone of the opposite response along the x-axis for a minimum of 20 samples, and at least 20 samples were on the opposite side of the horizontal midline from the ultimately executed response (Figure 2A). We also calculated the distance traveled by the response finger in each trial as the sum of the linear distance between each consecutive sample from the first to the 101st sample. 

Figure 2

 

(A) Mean movement trajectories from Experiment 1 plotted in two-dimensional (horizontal and vertical) space for the close box condition and the far box condition. All trials where the left-side target was selected were flipped over the vertical axis for this figure such that all movements are plotted as going to the right side. Plotted separately are change-of-mind responses (red) and non-change-of-mind responses (green) on low-coherence trials and all responses on high-coherence trials (black). At each point, a line is drawn to reflect ± 1 SEM in the horizontal and vertical directions. (B) Change-of-mind frequency and (C) initiation latency for the close and far conditions on low-coherence trials in Experiment 1. Error bars reflect a confidence interval from the within-subject error term (e.g., Loftus & Masson, 1994). (D) Change-of-mind frequency for each participant with participants ordered according to the magnitude of the change-of-mind frequency difference between close and far targets with the largest magnitude difference on the left.

Additionally, we calculated the peak deviation of movement in each trial. To calculate this measure, we first used the mean resampled movement trajectory for high-coherence trials at each of the four possible target locations (left vs. right, close vs. far response box conditions) as a baseline measure. Next, for each low-coherence trial, we calculated the difference along the x-axis for each point in the resampled trajectory relative to that baseline with a positive number indicating deviation toward the competing, nonselected response option. We calculated this difference only along the x-axis because the two competing targets did not differ on their y-axis or z-axis position relative to the starting position. Of the entire resampled trajectory, the point with the maximum pull in the x dimension toward the competing response option was defined as the point of peak deviation, reported in centimeters. 

Each individual trial was visually inspected (e.g., Song & Nakayama, 2008; Song & Nakayama, 2006; Song & Nakayama, 2007a, 2007b); for trials in which the default threshold clearly missed part of the movement or included substantial movement back to the starting point, thresholds were adjusted manually to more appropriate levels for that trial (1.2% of all trials). In addition, we removed all trials in which a large number of movement samples were dropped due to computer error or no movement was executed (1.1% of all trials). One participant was removed from the 2 × 2 ANOVA examining movement time and distance traveled on change-of-mind trials because that participant had no changes of mind in one of the two response box conditions. 

Results

Figure 2A shows a visual depiction of change-of-mind (red lines) and non-change-of-mind (green lines) responses with mean movement trajectories averaged across all participants plotted separately for the close target (i.e., the upper target, closer to the vertical meridian) and the far target (i.e., the lower target, farther from the vertical meridian). Mean trajectories for high-coherence trials are plotted in black for reference. Movement trajectories in trials in which the participant reached to the left-side target are flipped over the vertical meridian such that all movement trajectories are viewed as going toward the right-side target. We note here that the response boxes were equidistant from the starting position in both the close and far response box conditions; it is the distance between the response boxes themselves, not the distance from the participant's starting position, which varies between these conditions. 

We expected very few changes of mind in the high-coherence condition because the decision should be very easy, reflected in high overall accuracy rates (99.8% accurate; Figure 2, black lines). We found few overall changes of mind in the high-coherence condition (<1%) and no difference between the close and far response box conditions in change-of-mind frequency, t(13) = 1.19, p > 0.1. This confirms that our criteria for defining changes-of-mind criteria were reasonable. 

For low-coherence, non-change-of-mind trials, movement trajectories appear to travel a roughly similar distance in the close and far response box conditions (Figure 2A, green lines). As the response box distances from the starting position are equal, this is consistent with expectations. It is also consistent with expectations that movement trajectories on change-of-mind trials are initially directed toward the opposite response before being redirected toward the final response. However, importantly, change-of-mind movements appear to involve a steeper redirection after the initial movement, exhibiting greater peak deviation, in the far response box condition. Additionally, it appears that the hand has to travel a greater distance when a change-of-mind occurs in the far response box condition than when one occurs in the close response box condition. Because participants have to travel a greater distance in change-of-mind trials in the far response box condition, it likely also takes them a longer period of time to reach the target in trials in which they change their mind. Thus, it appears that changes of mind incur greater costs in terms of time and physical energy when the distance between competing responses is greater. 

To confirm this observation, we conducted 2 × 2 ANOVAs with factors of response type (change of mind vs. non-change of mind) and response box distance (close vs. far) on movement time, distance traveled (see Methods), peak deviation, and response accuracy. First, we focus on movement time. Not surprisingly, movement time was longer in trials in which the participant changed his or her mind (587 ms) relative to trials in which no change occurred (469 ms), F(1, 12) = 7.86 p < 0.05. Thus, the hand took longer to reach the target following movement initiation in trials in which the observer initially directed a movement toward one response but ultimately redirected to the opposite response. However, this cost varied depending upon the response box distance as revealed by a significant interaction, F(1, 12) = 5.72, p < 0.05. Specifically, in change-of-mind trials, movement times were longer for the far response box condition (605 ms) relative to the close response box condition (569 ms), F(1, 12) = 12.46, p < 0.01. However, there was no difference in movement time between the two response box conditions in trials in which there was no change of mind, F(1, 12) < 1. These results indicate that changes of mind incurred greater costs in the time taken to complete a movement when the response boxes were placed far apart relative to when they were placed close together. 

The same pattern of results was obtained for measurements of the distance traveled by the hand in each trial. As we expected, the hand traveled a greater distance in change-of-mind trials (33.6 cm) relative to trials in which no change of mind occurred (25.3 cm), F(1, 12) = 208.8, p < 0.001. In accord with our prediction, we observed that this difference varied depending on the distance between response boxes, F(1, 12) = 97.82, p < 0.001. Specifically, we observed that when a change of mind did not occur, the distance traveled was equivalent regardless of the response box distance, F(1, 12) = 1.4, p > 0.1 (close: 25 cm, far: 25.7 cm). This was expected because the distance from the starting position to each box was equivalent regardless of whether the response boxes were close together or far apart. However, in change-of-mind trials, the distance traveled was much greater when the boxes were far apart (36.7 cm) than when they were close together (30.5 cm), F(1, 12) = 121.85, p < 0.001. This confirms the observations made in Figure 2A; for change-of-mind trials (red lines), the distance traveled by the hand is greater when the response boxes were far apart relative to when they were close together. Thus, as with movement time, motor costs measured by the distance traveled during change-of-mind responses were much greater when the response boxes were far apart. In this case, these are presumed to be physical energy costs as the hand had to travel a longer distance when the response boxes were far apart. 

Peak deviation, too, exhibited similar outcomes. The peak deviation measure used in the present study reflects the maximum point of deviation relative to baseline for each reach trajectory, thus giving a sense of how far the participant moved his or her hand toward the opposite response option before correcting the movement midstream (see Methods for more details). Peak deviation was greater in change-of-mind trials (9.03 cm) than in non-change-of-mind trials (1.83 cm), F(1, 12) = 156.16, p < 0.001. This reflects greater pull toward the competing response option in change-of-mind trials as expected given the way in which changes of mind are operationally defined in the present study. However, as with other measures, the effect of changes of mind on peak deviation was affected by response box distance, F(1, 12) = 57.41, p < 0.001. In change-of-mind trials, peak deviation was greater when the response boxes were far apart (11.34 cm) relative to when they were closer together (6.72 cm), F(1, 12) = 146.7, p < 0.001. This indicates that the pull toward the competing response was greater when the two response boxes were farther apart in space, indicating that a steeper redirection of movement is required in change-of-mind responses in the far response box condition. There was no difference in peak deviation between close and far response box conditions when no change of mind occurred, F(1, 12) = 1.21, p > 0.1. Response accuracy was largely unaffected by whether the observer changed his or her mind about a decision and by the distance between response options (ps > 0.1). 

These analyses establish that change-of-mind motor costs are greater in the far response box condition relative to the close response box condition. For the critical analysis (Figure 2B), we examined change-of-mind frequency according to response box distance. If perceptual decision-making processes are adjusted to account for the costs associated with change-of-mind trials, we expect changes of mind to occur less often when the response boxes are far apart, and thus, the motor costs of a change of mind are high. Indeed, we did find that changes of mind occurred less often when the response boxes were far apart (3.6%) than when they were close together (8.3%), t(13) = 3.09, p < 0.01. In other words, when expected motor costs of a change of mind were high because the response options were far apart, observers changed their mind less often. This outcome supports the notion that perceptual decision processes do account for expected change-of-mind motor costs. Change-of-mind frequencies for individual participants are shown in Figure 2D; 13 of 14 participants changed their mind less frequently when the response boxes were further apart. Additionally, the magnitude of initiation latency was longer when the boxes were far apart (739 ms) than when they were close together (724 ms; Figure 2C), but this effect did not approach significance, t(13) < 1. We discuss this analysis in more detail in the results of Experiment 2. 

We applied relatively conservative criteria to define change-of-mind trials to ensure that the seemingly counterintuitive integration of motor costs into perceptual decision making is a robust phenomenon. These appeared to be appropriate criteria for defining changes of mind in a visually guided reaching task as we showed that changes of mind were very infrequent in the high-coherence condition in which decisions should have been trivially easy for participants. Researchers in recent years have used a multitude of ways to define a change-of-mind response (e.g., Albantakis & Deco, 2011; Moher & Song, 2013; Resulaj et al., 2009; Song & Nakayama, 2008), and eye movements, too, display patterns consistent with competing activation of multiple responses in parallel reflecting a change of mind after an initial decision (e.g., Arai, McPeek, & Keller, 2003; McPeek, Han, & Keller, 2003). The present method is advantageous in that change-of-mind boundaries are set separately for each individual participant, mitigating for interparticipant differences in their default motor movements (e.g., Albantakis, Branzi, Costa, & Deco, 2012). 

Together, the results of Experiment 1 support our hypothesis that perceptual decision-making processes account for the expected motor costs that a change of mind will incur. When change-of-mind motor costs are high because two competing response options are far apart in space, changes of mind occur less often in a perceptual decision-making task. When the costs are less severe because the response options are closer together, changes of mind are more frequent. 

Experiment 2: Dynamic adjustments in decision strategies to avoid costly changes of mind

In Experiment 1, we found evidence that observers were able to modulate their perceptual decision-making processes based on the expected motor costs associated with changes of mind. However, the distance between response boxes was consistent over the course of each block. Thus, it may be the case that changes in perceptual decision-making processes based on expected change-of-mind motor costs take significant time to implement. 

In Experiment 2, we tested whether changes in perceptual decision strategies might be dynamically adjusted by varying response box distance on a trial-by-trial basis. The response boxes were presented only 1 s before each trial began, providing observers only a brief period of time to adjust their decision processes based on expected costs associated with changes of mind. If observers are able to adjust their decision processes dynamically, we expect to see similar results to Experiment 1. 

Methods

Eleven Brown University undergraduates (seven male, mean age = 19 years) participated in a session lasting approximately 1 hr in exchange for course credit. All participants were right-handed with normal or corrected-to-normal color vision. The protocol was approved by the Brown University Institutional Review Board. 

The methods for Experiment 2 were identical to Experiment 1 except in one respect. The distance between the response boxes varied randomly from trial to trial between close and far. After each trial, the response boxes disappeared during the intertrial interval and reappeared with the fixation cross of the next trial. After 1 s, the moving dot display then appeared. In Experiment 1, the distance varied from block to block. Each participant completed at least seven out of a maximum of 10 blocks, and the average number of blocks completed was 9.4. 

Each individual trial was visually inspected (e.g., Song & Nakayama, 2008; Song & Nakayama, 2006; Song & Nakayama, 2007a, 2007b); for trials in which the default threshold clearly missed part of the movement or included substantial movement back to the starting point, thresholds were adjusted manually to more appropriate levels for that trial (1.1% of all trials). In addition, we removed all trials in which a large number of movement samples were dropped due to computer error or because no movement was executed (2.0% of all trials). 

Results

Figure 3A presents a visual depiction of reach movement trajectories across experimental conditions of interest. As in Experiment 1, mean movement trajectories are plotted for change-of-mind trials (red lines) and non-change-of-mind trials (green lines) on low-coherence trials along with mean trajectories for all high-coherence trials for reference (black lines). 

Figure 3

 

(A) Mean movement trajectories from Experiment 2. Trajectories are plotted in two-dimensional (horizontal and vertical) space for the close box condition and the far box condition. Plotted separately are change-of-mind responses (red) and non-change-of-mind responses (green) on low-coherence trials and all responses on high-coherence trials (black). All trials where the left-side target was selected were flipped over the vertical axis for this figure such that all movements are plotted as going to the right side. At each point, a line is drawn to reflect ± 1 SEM in the horizontal and vertical directions. (B) Change-of-mind frequency and (C) initiation latency for the close and far conditions on low-coherence trials in Experiment 2. Error bars reflect a confidence interval from the within-subject error term (e.g., Loftus & Masson, 1994). (D) Change-of-mind frequency for each participant with participants ordered according to the magnitude of the change-of-mind frequency difference between close and far targets with the largest magnitude difference on the left.

We initially examined only high-coherence trials (Figure 3A, black lines) to again determine whether our metric for defining changes of mind was sufficiently conservative such that changes of mind occurred very infrequently when the task was easy. As in Experiment 1, changes of mind occurred on fewer than 1% of high-coherence trials, and there was no difference in frequency between the close and far conditions, t(10) < 1. 

The trajectories for low-coherence trials in Figure 3A look markedly similar to those of Experiment 1 (Figure 2A), which is not entirely surprising because the same criteria were used to define changes of mind in both experiments. That is, in non-change-of-mind trials (green lines), trajectories appear roughly similar whether the response boxes are close or far apart in space. However, trajectories in change-of-mind trials (red lines) in the far response box condition appear to travel a greater distance and exhibit a steeper redirection than changes of mind in the close response box condition. As in Experiment 1, this indicates that changes of mind are more costly in terms of time and physical energy when the competing response options are farther apart in space. 

Statistical analyses showed that the results observed in Experiment 1 were largely replicated in Experiment 2. Movement times were longer on change-of-mind responses than on nonchange responses (631 ms vs. 471 ms), F(1, 10) = 67.3, p < 0.001. However, again we found that response box distance affected the magnitude of this effect, F(1, 10) = 15.13, p < 0.01. In change-of-mind trials, movement times were longer when the response boxes were far apart (660 ms) relative to when they were close together (603 ms), F(1, 10) = 26.36, p < 0.001. There was no difference in movement time between close and far response box conditions for non-change-of-mind trials, F(1, 10) < 1. Thus, as in Experiment 1, changes of mind incurred greater time costs when the response boxes were far apart relative to when they were close together. 

Consistent with Experiment 1, a greater distance was traveled in change-of-mind trials compared to nonchange trials (33.4 cm vs. 24.8 cm), F(1, 10) = 119.96, p < 0.001. As with movement times, this difference was mediated by whether the response boxes were close together or far apart, F(1, 10) = 26.36, p < 0.001. In change-of-mind trials, participants traveled a greater distance with their movement when the boxes were far apart (35.6 cm) than when they were close together (31.3), F(1, 10) = 19.69, p < 0.01. Again, this confirms the observations made in Figure 3A. Movement trajectories in trials in which the observer changed his or her mind (red lines) traveled a greater distance when the boxes were far apart relative to when they were close together. There was no difference in distance traveled on nonchange trials, F(1, 10) < 1. 

The peak deviation measure again reflected the same pattern. Peak deviation was greater in change-of-mind trials (9.55 cm) relative to non-change-of-mind trials (1.59 cm), F(1, 10) = 125.28, p < 0.001, reflecting an increased deviation in movement trajectories in trials in which the participant changed his or her mind. Furthermore, the effect of response box distance on peak deviation was mediated by whether a change of mind occurred, F(1, 10) = 14.44, p < 0.01. In change-of-mind trials, peak deviation was greater in the far response box condition (11.22 cm) relative to the close response box condition (7.87), F(1, 10) = 30.87, p < 0.001. There was no difference in non-change-of-mind trials, F(1, 10) < 1. Thus, there was a greater pull in the direction of the opposite response in change-of-mind trials when that competing response was farther away in space. There were no main effects or interactions with respect to accuracy, ps > 0.1, again suggesting that changes of mind and response box distance did not affect overall performance accuracy. 

As in Experiment 1, we determined whether the frequency of changes of mind decreased when the boxes were far apart, reflecting a strategic adjustment in perceptual decision processes to avoid changes of mind when they incurred high motor costs (Figure 3B). We again found a decreased frequency of changes of mind when the response boxes were far apart (3.0%) relative to when they were close together (4.4%), t(10) = 2.6, p < 0.05. Nine of 11 participants changed their mind less frequently when the response boxes were far apart (Figure 3D). 

Thus, not only do perceptual decision processes account for the expected cost of a change of mind by reducing the frequency of changes of mind when motor costs incurred by changes of mind are high, but they can do so dynamically as the response box distance was made available to the observer only 1 s before each trial began. This result highlights surprising flexibility in the degree to which perceptual decision-making processes account for expected change-of-mind motor costs. 

Notably, the magnitude of the difference in change-of-mind frequency between the close and far target conditions was more than three times greater in Experiment 1 (4.9%; Figure 2B) relative to Experiment 2 (1.4%; see Figure 3B, Table 1). A 2 × 2 mixed ANOVA with experiment as a between-subject factor, response box distance as a within-subject factor, and change-of-mind frequency as the dependent variable revealed that this interaction approached significance, F(1, 23) = 3.41, p = 0.078. Although this difference should be interpreted with caution because the interaction only approached significance, the difference is noteworthy because it strongly implies that adjustments in perceptual decision strategies are more difficult to implement in dynamic environments in which motor costs related to change-of-mind responses are constantly changing. This reduced difference in mixed trials compared to blocked trials suggests that the more frequent occurrence of changes of mind in the close compared to the far response box condition is not merely a result of our classification criteria. For example, if ambiguous movements were more likely to be classified as changes of mind in the close condition than in the far condition due to the proximity of the competing response options in the close condition to the center, we would not expect a difference between blocked and mixed trials. 

Table 1

 

Data from low-coherence trials reported separately for change-of-mind and non-change-of-mind responses and data for movement time, distance traveled, and peak deviation reported separately for change-of-mind and non-change-of-mind responses. Notes: Error terms reflect standard error of the mean.

	Change of mind?	Experiment 1		Experiment 2
		Close	Far	Close	Far
Initiation latency	All trials included	724 ± 62 ms	739 ± 57 ms	904 ± 96 ms	928 ± 95 ms
Change-of-mind frequency	All trials included	8.3 ± 1.7%	3.6 ± 0.5%	4.4 ± 1.0%	3.0 ± 0.6%
Movement time	Yes	569 ± 16 ms	605 ± 14 ms	603 ± 26 ms	660 ± 30 ms
	No	467 ± 11 ms	471 ± 12 ms	468 ± 16 ms	475 ± 15 ms
Distance traveled	Yes	30.5 ± 0.4 cm	36.7 ± 0.8 cm	31.3 ± 0.8 cm	35.6 ± 1.2 cm
	No	25.0 ± 0.2 cm	25.7 ± 0.3 cm	24.6 ± 0.2 cm	25.0 ± 0.3 cm
Peak deviation	Yes	6.7 ± 0.4 cm	11.3 ± 0.8 cm	7.9 ± 0.6 cm	11.2 ± 0.9 cm
	No	1.6 ± 0.1 cm	2.0 ± 0.1 cm	1.5 ± 0.1 cm	1.7 ± 0.1 cm

Overall initiation latency was longer when response boxes were far apart (928 ms) relative to when they were close together (904 ms; Figure 3C), t(10) = 2.6, p < 0.05. This result suggests a plausible mechanism by which a reduction in changes of mind is implemented. Specifically, observers may wait for more evidence before initiating responses when the response boxes are far apart. By doing this, they will be more certain of their initial response and thus reduce the likelihood that they will change their mind after the initial motor plan is initiated. Although the same pattern of results was observed in Experiment 1, it did not reach significance there. This discrepancy may be because recent experience plays a key role in determining motor costs (e.g., Song and Nakayama, 2007). Perhaps more efficient strategies for reducing motor costs associated with changes of mind are available when trial types are blocked, and thus, the immediately preceding trials are typically of the same type as the current trial. 

Together, these data again demonstrate that perceptual decision processes adapt to avoid changes of mind when anticipated motor costs of those changes of mind are high. This was true even when response box distance was randomly varied from trial to trial, suggesting that adjustments in the decision process to avoid costly changes of mind were implemented dynamically. 

General discussion

Previous research has shown that decision making involves not only consideration of higher-level goals, but also of the costs associated with the motor outputs of competing choices (e.g., Cos et al., 2011; Morasso, 1981; Sabes & Jordan, 1997). In the present study, we expanded on this research to examine whether motor costs associated with changes of mind could be incorporated into decision-making processes. When response boxes were far apart, and thus the motor costs incurred by a change of mind would be high, participants changed their mind less frequently in a perceptual decision-making task. These were highly complex motor costs that were considered. That is, not the motor cost of the initial choice, which was equivalent between the two response box conditions, but the motor cost of switching decisions after making an initial choice, a cost that varied depending on the distance between two competing response options. Finally, in Experiment 2, we found these adjustments occurred even when the response box distance varied randomly from trial to trial and was only available to the participant for 1 s before stimulus onset. Thus, adjustments in perceptual decision processes based on anticipated motor costs are highly flexible. 

In both experiments, initiation latency was longer when the response boxes were farther apart than when they were close together. This suggests a mechanism by which a reduction in changes of mind may be implemented. That is, participants may have waited for more evidence before initiating a movement when the boxes were placed far apart relative to when they were close together. By doing this, they could be slightly more certain of their response and thus less likely to change their mind after a motor movement was initiated. It is notable that the difference in initiation latency only reached significance in Experiment 2. Thus, when response box distance was blocked, creating a more stable task environment, participants may have engaged alternative mechanisms to reduce change-of-mind frequency that had weaker effects on initiation latency. 

The data from both experiments are consistent with the distributed consensus model (Cisek, 2012), which suggests that both potential motor outputs and higher-level goals are considered in decision processes. Previous research has shown that decision processes are impacted by the motor costs associated with different action options (e.g., Cos et al., 2011). The present study shows even stronger connections between anticipated motor outputs and decision making, whereby perceptual decision processes account for the motor costs of switching from one decision to another and reduce the frequency of these changes of mind when those motor costs are high. 

A number of previous studies have investigated how action costs can be efficiently incorporated into perceptual decision-making processes. For example, observers adjust decision strategies to account for asymmetric rewards between competing response options (e.g., Fleming, Whiteley, Hulme, Sahani, & Dolan, 2010; Green & Swets, 1966; Rorie, Gao, McClelland, & Newsome, 2010; Summerfield & Koechlin, 2010; Whiteley & Sahani, 2008) or trade-offs in sensory and motor noise (e.g., Battaglia & Schrater, 2007; Faisal & Wolpert, 2009). Drift diffusion modeling (e.g., Ratcliff, 1978) has proven to be a useful tool for posing possible mechanisms by which decision making is adjusted to account for these kinds of costs and benefits associated with competing decision options. For example, decision biases might be encoded either by changing the starting point of evidence accumulation toward the boundary of the more valuable option or by changing the rate of the accumulation of sensory evidence in favor of one response or another (i.e., the drift rate). Neurophysiological evidence from cellular recordings in monkeys (Rorie et al., 2010) and functional neuroimaging in humans (Summerfield & Koechlin, 2010) is more consistent with changes in the starting point of evidence as the mechanism by which responses are biased toward a more valuable option. These studies suggest the possibility that decision strategies, at least in some cases, are adjusted before evidence accumulation begins. 

A similar framework may prove useful when applied to the data presented here. According to Resulaj et al.'s (2009) modified drift diffusion model, a motor plan is initiated when evidence in favor of one response crosses an initial threshold. However, evidence continues to be evaluated, and if a change-of-mind threshold for the competing response is crossed, a second motor plan is triggered toward that competing response. This model might prove useful in explaining the reduction of changes of mind in the far response box condition reported here. For example, when anticipated motor costs for a change of mind are high, the threshold of evidence for the initial decision might be increased. As a result, initiation latency would be increased because a greater amount of evidence would be required to initiate a movement, and changes of mind would decrease because a greater amount of evidence would be required to reverse the initial decision. Both of these outcomes are consistent with our data. Future research may benefit from applying this approach to illuminate the mechanisms by which changes of mind are reduced when anticipated motor costs are high and whether adjustments occur before evidence is accumulated or while a decision is being made. 

Temporal aspects of change-of-mind responses

In the present study, perceptual input was extinguished the moment that movement onset occurred in each trial. Thus, changes of mind in the present study do not reflect a decision to redirect hand movements based on sensory evidence acquired after the initial movement already began. In a study examining similarly redirected movements, Song and Nakayama (2008) argued that observers were continuously evaluating evidence for competing decisions in parallel because the time between movement initiation and movement redirection (i.e., change-of-mind occurrence) was shorter than movement initiation latency. In contrast, they showed that in a forced double-step task in which a change of mind was necessarily triggered by stimulus input that occurred concurrent with movement initiation, these two times were equivalent. Song and Nakayama (2008) suggested that changes of mind thus occur when an initial decision reaches threshold first and is executed but a second decision reaches threshold shortly thereafter, overriding the initial motor plan. 

We observed a similar outcome; the time between movement initiation and movement redirection (i.e., the point immediately following peak deviation at which the movement is redirected toward the opposite response) on low-coherence change-of-mind trials (Experiment 1: 296 ms; Experiment 2: 330 ms) was much shorter than mean initiation latency even in easier, high-coherence, non-change-of-mind trials (Experiment 1: 522 ms; Experiment 2: 554 ms). The short correction time relative to longer initiation latencies suggests that changes of mind occurred because evidence in favor of competing options was being evaluated in parallel and not because participants decided to redirect movements after the movement had already been initiated. This is also consistent with neurophysiological evidence demonstrating that multiple motor plans for reaches (Cisek & Kalaska, 2002, 2005) and saccades (e.g., McPeek et al., 2000; McPeek & Keller, 2002) can be implemented in parallel without a substantial delay. 

Broader implications for understanding decision making

The results from the present study have broad implications for understanding the decision-making process. Together with other recent work (e.g., Cisek, 2012; Cos et al., 2011; Morasso, 1981; Sabes & Jordan, 1997), the data presented here highlight the degree to which anticipated motor costs are accounted for in decision-making processes. In the present study, we discovered that decision processes were able to account for highly complex anticipated motor costs. This was not merely a matter of determining which of two competing response options required more time and physical energy. Rather, participants were anticipating the costs that would be incurred in the event that the initial decision was overruled, and this information was being considered as part of a perceptual decision-making process. The fact that this type of forward-thinking and subtle adjustment occurs in a perceptual decision-making task suggests that even simple decisions involve sophisticated calculations that extend well beyond a mere evaluation of sensory evidence. Continued use of this combined perception-action approach has promising potential to lead to new insights into how and why people make decisions that require action outputs in a variety of contexts, ranging from simple perceptual decisions to more complex decisions, such as what to eat, what to wear, or even who to approach at a bar, that may have important, long-lasting consequences. 

For example, the work presented here appears to contradict the good-based model of economic choice, which suggests that abstract representations of competing choice options are compared independently from any sensorimotor contingencies (e.g., Padoa-Schioppa, 2011). Still, it is possible that the degree to which action costs are incorporated into decision-making processes varies depending on the type of decision. Further research is needed to fully understand these interactions and to determine if anticipated motor costs affect higher-order decision processes, such as preference or economic decision making. 

The present study provides one factor, anticipated motor costs, which affects the likelihood of a change of mind. However, there is still a great deal more to uncover in order to understand what factors trigger change-of-mind responses. For example, in a recent study (Moher & Song, 2013) we found that “partial error” responses, similar to the change-of-mind responses studied here, tend to be repeated when the task context (in that case, target color) is also repeated. A greater description of the causes of change-of-mind responses in laboratory tasks such as these may shed light on how and why people change their mind about decisions in everyday behavior. 

Conclusions

Participants changed their mind about a perceptual decision less frequently when motor costs associated with changes of mind were high because competing response options were far apart in space. These results highlight strong connections between decision processes and anticipated motor output, with dynamic adjustments in decision-making processes implemented to avoid changes of mind when the associated motor costs are high. 

Acknowledgments

This project is supported by a Brown University Salomon faculty research award and NIGMS-NIH (P20GM103645) to J.-H. S., and the Center for Vision Research fellowship and the Brown Training Program in Systems and Behavioral Neuroscience NIH T32MH019118 to J. M. 

Commercial relationships: none. 

Corresponding author: Jeff Moher. 

Email: [e-mail]. 

Address: Cognitive, Linguistic, and Psychological Sciences, Brown University, Providence, RI, USA.