Geometric structure and chunking in reproduction of motion sequences

Abstract

Learning by imitation is fundamental to human behavior, but not all observed actions are equally easy to imitate. To understand why some actions are more difficult to imitate than others, we examined how higher-order relationships among the components of a stimulus model influenced the fidelity with which an action could be observed and then reproduced. With static contours, perception and short-term memory are strongly influenced by contour geometry, particularly by the presence and distribution of curvature extrema. To determine whether analogous relationships among subcomponents of a seen action would be important in encoding the action for subsequent reproduction, we manipulated actions' spatio-temporal geometry. In three experiments, we measured imitation fidelity for sequences of randomly directed, linked motions of a disc. The geometry of the disc's motion path strongly affected the accuracy of subsequent imitation: When the disc moved along a trajectory whose components were fairly consistent in their directions, imitation was strikingly better than when with irregular, jagged trajectories. A second experiment showed that this effect depended not upon co-variation in stimulus models' spatial extent, but rather on the relationship between successive movement directions. In a final, learning experiment, subjects had multiple opportunities to view and reproduce each model. The effect of the model's geometry persisted throughout the learning process, suggesting that it does not depend upon variables such as familiarity or expectancy but is somehow inherent to the pattern generated by the disc's motion. Our findings suggest that when analyzing seen actions, the brain privileges regular, consistent curvatures, grouping components that form a coherent shape into a unified “chunk.” Inconsistencies among the directional components of a motion sequence cause the sequence to be chunked into additional components, which increases the load on working memory, undermining the fidelity with which the sequence can be imitated.

Introduction

In daily life, behavior often is guided by dynamic events that reveal themselves piecemeal as they unfold over time. In initial stages of processing such an event, the spatio-temporal stream of information on the retina is segmented into appropriate parts, or “packets” (Shipley & Maguire, in press; Shipley, Maguire, & Brumberg, 2004; Zacks & Tversky, 2001). Then for purposes of understanding or recognizing the event, segmentation may be followed by the enactment of a neural representation in which individual segments are recoded or “chunked” into a more compact, robust form (Gobet et al., 2001; Miller, 1956). Segmentation and recoding may be particularly important when the dynamic event is some complex, multi-component action that must be observed, remembered, and later reproduced. 

Although much empirical and theoretical work has focused on the processing and parsing of static shapes (e.g., Attneave, 1954; Feldman & Singh, 2005; Hoffman & Richards, 1984; Richards, Dawson, & Whittington, 1986), little is known about the treatment of dynamic stimuli. The processing of static shapes and processing of dynamic events differ in a variety of ways (e.g., Bex, Simmers, & Dakin, 2001, 2003; Brown & Voth, 1937; Caelli & Dodwell, 1980; Viviani & Stucchi, 1989, 1992), but there is some evidence of parallels between the two. For example, the trajectory of a moving object and the orientation of a static contour have similar representations at the level of primary visual cortex (Jancke, 2000). Also, at a behavioral level, motion trajectories and static contours seem to be segmented in similar ways, i.e., based on their curvature extrema (Shipley & Maguire, in press; Shipley et al., 2004). These parallels led us to ask whether factors like curvature extrema would also influence the ease with which dynamic events were processed for subsequent imitation. For a direct test of this hypothesis, we systematically varied the spatio-temporal structure of stimulus models and measured the fidelity with which they were imitated. We reasoned that explicitly controlling the geometric structure of an action's trajectory could promote or discourage multiple-part representations of the trajectory. Specifically, we examined how the directional relationship between component segments in our stimulus models affected accuracy and error patterns during subsequent imitation. 

Our experiments used a delayed imitation paradigm (Agam, Bullock, & Sekuler, 2005; Agam, Galperin, Gold, & Sekuler, 2007; Sekuler, Siddiqui, Goyal, & Rajan, 2003) in which subjects first view a disc moving along a set of randomly oriented, linked, straight motion segments. Shortly after the disc disappears from sight, the subjects are asked to reproduce from memory the motion path they had just seen, using a stylus on a graphic tablet. In one study (Agam et al., 2007), as particular complex motion sequences were repeated, improvement in performance was accompanied by chunking, as revealed by the timing and errors associated with successive imitations of the same model. If chunking facilitated imitation learning over repeated encounters with the same stimulus model, we hypothesized that the very first imitation of a mutli-component behavior would be aided if the model behavior's own components promoted recoding of the model into chunks. Another study with this paradigm (Agam et al., 2005) produced some evidence that model trajectories with a similar number of components exhibited considerable heterogeneity in the accuracy with which they were imitated, suggesting that some models can be encoded more efficiently than others. Although we did not systematically vary the geometry of the trajectories, we hypothesized that this heterogeneity arose from the differential “chunkability” of various trajectories. 

In Experiment 1, we systematically manipulated models' structure by varying the pattern of direction changes making up the model trajectory. Specifically, we controlled the rules governing clockwise (CW) or counterclockwise (CCW) changes in the disc's motion. To preview our results, we found that generally, a relatively enclosed, convex trajectory, along which the disc moved consistently CW (or consistently CCW), was easier to imitate than a “zig-zag” trajectory, in which the disc toggled between CW and CCW motions. Experiment 2 controlled the distance between the segment components of the models, so that some models comprised spatially compact trajectories, whereas other models spanned a larger display area. Control over this characteristic of the models enabled us to dissociate the impact of both angular and distance relationships among the individual segments in models. Experiment 3 tested whether the effect of model structural relationships, demonstrated in Experiments 1 and 2, was related to stimulus predictability, which is known to affect motion detection and discrimination (Ball & Sekuler, 1980, 1981; Sekuler, Sekuler, & Sekuler, 1990). Subjects were shown the same motion sequence multiple times, rendering models more familiar and thereby reducing stimulus uncertainty. Thus, we were able to measure performance under different degrees of familiarity and predictability, while structural relationships were held constant. 

Experiment 1

The purpose of Experiment 1 was to examine how structural relationships in motion sequences, specifically the presence or absence of curvature extrema, alter subjects' ability to memorize and later reproduce those sequences. As will be explained below, stimulus models differed in the degree of consistency in CW or CCW motions comprising each model. 

Methods

Twelve normal subjects, ages 18–23 (10 female), participated in the experiment after providing written informed consent. We used an imitation task similar to that described in Agam et al. (2005, 2007). Figure 1A shows a schematic diagram of the task. Each quasi-random motion sequence was generated by the steady movement of a yellow disc (1 deg visual angle in diameter) against a black background on a computer screen, which subjects viewed from a distance of 57 cm. The disc, which was initially positioned at the center of the screen, moved along a series of five connected, straight segments, each 1.5 deg visual angle long. Each segment took 525 ms to complete and was followed by a 225-ms pause (stationary disc) before the next segment. As the disc moved, it left no visible trail, so subjects had to knit together the disc's trajectory in their mind's eye and hold it in working memory. The motion segments' directions were randomized under two constraints. First, to minimize verbal encoding of the shape implied by the disc's motions (Sekuler et al., 2003), segments were kept from intersecting or approaching one another closer than half the length of a segment. Second, the angle between consecutive segments was constrained to be between 30 and 150 degrees. In addition to these two constraints, which were used in previous experiments, we explicitly controlled the direction of rotation—CW or CCW—between adjacent segments. The following provides a formal description of the stimulus generation process: θ₁, the orientation of S₁, the first segment, was chosen randomly from a uniform distribution of orientations: 

$0°\le {\theta }_1\le 360°.$

(1)

For succeeding segments, S₂ to S₅, each segment's orientation is given by 

${\theta }_n={\theta }_{n-1}+\Delta \theta ,30°|\Delta \theta |\le 150°,$

(2)

where Δθ is chosen randomly from a uniform distribution. The orientation difference between segments S₁ and S₂ establishes a local CW or CCW trend in the trajectory. For each subsequent segment S_n (n = 3, 4, 5), we designate with a binary variable κ_n−2 whether that segment's orientation conforms to the trend set by the previous two segments (segment is curvature-consistent) or violates the trend (segment is curvature-inconsistent). For the third segment, then 

${\kappa }_1=\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}({\theta }_1-{\theta }_2)\times \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}({\theta }_2-{\theta }_3),$

(3)

where κ_n = 1 represents consistency with the previous CW or CCW trend and κ_n = −1 represents a violation of the trend. Similarly, 

${\kappa }_2=\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}({\theta }_2-{\theta }_3)\times \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}({\theta }_3-{\theta }_4),$

(4)

 

${\kappa }_3=\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}({\theta }_3-{\theta }_4)\times \mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}({\theta }_4-{\theta }_5).$

(5)

 

Figure 1

 

Task and Stimuli. (A) Schematic diagram of the experimental paradigm. Note that during stimulus presentation and imitation, the disc did not leave a trace, so subjects only saw its instantaneous position; the dashed lines are for illustration purposes. (B) Example stimulus models for each of the eight possible stimulus types in Experiment 1. The dotted circles denote the starting point of the trajectories. A plus sign indicates consistence with the directional trend set by the previous two segments, and a minus sign indicates a violation of that trend. (C) Example stimulus models for Experiment 2. Note that each model in panels B and C is only one of many existing possibilities that were used in the experiments.

Therefore, the curvature consistency of every five-segment stimulus model can be characterized using a three-element vector, K = { κ ₁, κ ₂, κ ₃}, indicating the degree to which the disc's motion changes its course from CW to CCW and vice versa, with {1, 1, 1} defining an enclosed pattern and {−1, −1, −1} a constant “zig-zag.” For convenience, from this point on we shall use only the sign of K, so that models will be classified as +++, −+−, −−−, etc. It is important to emphasize that for each value of K, there exist many possible models; K defines the general relationship between segments but does not explicitly constrain their orientations, so on each trial a completely novel sequence was presented. Figure 1B shows sample stimulus models for each one of the eight (2 ³) possible stimulus types. 

Note that the literature offers several terms to describe the curvature in contours or trajectories. Some authors use convexity vs. concavity (e.g., Barenholtz, Cohen, Feldman, & Singh, 2003; Bertamini, 2001) or positive extrema vs. negative extrema (e.g., Feldman & Singh, 2005; Richards et al., 1986) to describe the direction of curvature with respect to the interior of the object or the figure. As none of our stimulus trajectories formed a closed contour, the trajectories' “interior” is only weakly defined. Therefore, we shall use the terms curvature consistency and curvature inconsistency to describe changes from CW to CCW motion (or vice versa). Thus, a segment that conforms to the trend (CW or CCW) established by the two preceding segments will be referred to as a consistent segment, whereas an inconsistent segment will describe a segment violating the previous CW or CCW relationship. 

After completing its motion over the series of segments, the yellow disc disappeared, and a 3.75-s retention interval ensued. At the end of the retention interval, a blue disc appeared on the screen, prompting subjects to begin their imitation. Subjects reproduced the movement of the disc by drawing with a stylus on a graphic tablet positioned directly in front of their preferred hand. To begin an imitation, they touched the tablet with the stylus and started drawing what they remembered they had seen. The blue disc's movement on the screen was yoked with a 1:1 aspect ratio and spatial extent to the movement of the stylus on the tablet. To signal the end of an imitation, subjects lifted the stylus from the tablet, after which they were shown their attempted imitation superimposed on a static image of the entire path that had been traversed by the stimulus disc. The path traveled by the stylus was saved after each imitation for offine analysis. 

Results

Figure 2A shows orientation errors for individual segments in each of the eight experimental conditions of Experiment 1. Clearly, curvature consistency in the disc's trajectory profoundly affects the shape of the serial position curve representing the accuracy of the resulting imitation ( p < 0 .0001, two-way ANOVA). In the more consistent patterns, such as +++ and −++, the primacy effect is almost completely absent, whereas curves corresponding to the more inconsistent patterns, like −−− and +−−, are more reminiscent of the serial position curves ubiquitously reported in the serial recall literature (e.g., Agam et al., 2005, 2007; Avons, 1998; Crowder, 1970; Drewnowski, 1980; Drewnowski & Murdock, 1980; Haberlandt, Lawrence, Krohn, Bower, & Thomas, 2005). To gauge the overall effect of curvature inconsistencies on performance, we sorted the stimulus types into groups comprising sequences with no inconsistent segments (+++), one inconsistent segments (++−, +−+, −++), two inconsistent segments (+−−, −+−, −−+), and three inconsistent segments (−−−). As can be appreciated from Figure 2B, the number of curvature inconsistencies strongly affects imitation accuracy (p < 0.0015, one-way ANOVA). 

Figure 2

 

Experiment 1. (A) Serial position curves. Each panel shows orientation error as a function of segment serial position for a different experimental condition, as indicated. (B) Performance as a function of the number of curvature inconsistencies (“−” signs) in the sequences. Stimulus types were collapsed into groups comprising sequences with 0, 1, 2, or 3 inconsistencies. Inset shows mean error across all five segments. Error bars are within-subject SEM (Loftus & Masson, 1994).

The effect of the sequences' geometry is not confined just to the number of curvature inconsistencies: The data shown in Figure 2A suggest a correspondence between the position at which curvature consistency is violated and the resulting upward shift in the error curve. This can be appreciated by comparing pairs of stimulus types that differ at one element of the vector K. For example, note how the inconsistency between the fourth and fifth segments in ++− sequences leads to a fundamentally different curve than +++ sequences, where the fifth segment is consistent with all previous segments ( p < 0 .002, two-way ANOVA). 

For a more comprehensive analysis, rather than compare all possible pairs of stimulus types, we divided the stimulus types into two data sets in three different ways: First, we compared models in which the value of κ ₁ was 1, i.e., models where the third segment was consistent with the first two segments, to models with κ ₁ = −1, where the third segment was inconsistent. Similarly, we created two other data sets, based on the values of κ ₂ and κ ₃. Figure 3 shows serial position curves for the two groups in each data set. Two interesting points arise from the figure: First, just as imitation accuracy is usually better for earlier segments than later ones (the Primacy effect), performance seems to be more robust to curvature inconsistencies when they occur early: Whether the third segments is consistent with the direction of the first two (left panel) makes little difference to the shape of the serial position curve ( p > 0 .6, two-way ANOVA) and only weakly affects the mean orientation error across all segments ( p < 0 .1, paired t-test). Conversely, inconsistencies in the fourth (middle panel) and fifth (right panel) segments result in a significant change in the shape of the serial position curve ( p < 0 .02, p < 0 .001, respectively) and in an increase in overall error ( p < 0 .04 for both). The second observation is that the serial position curves tend to diverge at the point at which inconsistency arises. For example, in the middle panel of Figure 3, where data are sorted according to κ ₂, the curves differ the most at the middle position, whereas differences in κ ₃ (right panel) modulate the final portion of the curves. 

Figure 3

 

Experiment 1, results with trials sorted according to the serial position where curvature consistency was or was not violated, as indicated by the gray shading. Left panel: Trials in which the third segment took a turn in a different CW/CCW direction from that set by the first and second segments (turned CW when segments 1 and 2 created a CCW trend or vice versa, dashed line) or conformed to that direction (turned the same way, CW or CCW, as segment 2 had turned relative to segment 1, solid line). Middle panel: The fourth segment consistent or inconsistent with the trend set by segments 2 and 3. Right panel: The fifth segment consistent or inconsistent with the trend set by segments 3 and 4. Plots show orientation errors against serial position in each of the groups. Insets show mean error across all five segments. Error bars are within-subjects SEM.

Experiment 2

In Experiment 1, we varied models' structural relationships by controlling the way in which the disc changed its direction of motion from CW to CCW and vice versa. We found that consistency in that domain was a crucial factor in subjects' ability to remember and imitate the sequences. However, variation in curvature consistency produces other correlated spatial consequences: When the disc moves steadily along a CW or CCW contour, for example, it is likely to remain within a relatively restricted region of space; we can describe such a trajectory as compact. A “zig-zag” pattern, on the other hand would generate a more spread out trajectory, which can be described as extended. To quantify these differences, we examined 1,000 randomly generated models of each class. Table 1 shows for each model class the mean and standard deviation of the distance, in degrees of visual angle, from a model's starting point to the most distant point anywhere else in the trajectory. As expected, models of class +++ were the most compact, and models of class −−− were the most extended. This difference in models' spatial extent could lead to differences in the utilization of visual resources for perceptually encoding and remembering models of different types. Also, eye movements would be more frequent when models were more spread out, possibly impairing spatial memory (Postle, Idzikowski, Della Sala, Logie, & Baddeley, 2006). This made it imperative to evaluate the possibility that models' size, rather than their internal structure, produced the effects observed in Experiment 1. Therefore, in Experiment 2, we explicitly controlled the spatial extent of stimulus models. 

Table 1

 

Mean and standard deviation of the distance, in degrees of visual angle, from a model's starting point to the most distant point.

Model type	Maximum distance	Standard deviation
+++	2.88	0.70
++−	3.32	0.80
+−+	3.97	0.82
+−−	3.99	0.95
−++	3.95	0.69
−+−	4.50	0.77
−−+	4.31	0.85
−−−	4.77	0.84

Methods

Eleven normal subjects, ages 19–21 (7 male), none of whom had taken part in Experiment 1, participated in this experiment after providing written informed consent. The details of the imitation task were identical to Experiment 1, except for the use of different rules to generate the stimulus models. The experimental variable here was the distance allowed between individual segments in the shape outlined by the motion trajectory: The first two segments, S ₁ and S ₂, were generated according to the previously described quasi-random rules:  

$\begin{array}{c}0°\le {\theta }_1\le 360°\\ \\ {\theta }_2={\theta }_1+\Delta \theta ,30°\le |\Delta \theta |\le 150°\end{array}.$

(6)

 

From the third segment on, the Euclidean distance d between each point along segment S _n and points along segments S ₁,…, S _n−2 was forced to be within certain lower and upper limits. Those limits, expressed as multiples of segment length, l, differed between the three experimental conditions. In the Compact condition,  

$0.5l\le d\le 1.5l.$

(7)

In the Intermediate condition,  

$l\le d\le 3.5l.$

(8)

In the Extended condition,  

$1.5\le ld<\infty .$

(9)

The three conditions, then, represent the spatial extent of the disc's trajectory. Note that as in Experiment 1, many different models were used in each condition. Figure 1C shows sample models for each one of the conditions. To quantify the differences among the spatial extent of models in each of the three classes, we examined 1,000 randomly generated models in each condition. The mean distance (in degrees of visual angle) from the model's starting point to the most distant point along its trajectory was 3.08 ( SD 0.47), 4.37 (0.64), and 5.79 (0.82), for Compact, Intermediate, and Extended type models, respectively. The three model classes span about the same range of distances as that covered by the model classes in Experiment 1 and therefore comprise an appropriate control for possible influences of spatial extent. Stimulus types were randomly interleaved and appeared at equal probability. Each subject performed 360 trials in the course of two sessions. 

Results

Figure 4 shows accuracy results for Experiment 2. There was no significant difference between the conditions in terms of mean orientation error ( p > 0 .85, one-way ANOVA), nor was there an interaction between condition and serial position ( p > 0 .45, two-way ANOVA). This suggests that the outcome of Experiment 1 was not due to differences in models' spatial extent. 

Figure 4

 

Experiment 2, serial position curves. Solid, dashed, and dotted lines show orientation errors for each segment in the Compact, Intermediate, and Extended conditions, respectively. Inset shows mean error across all five segments. Error bars are within-subject SEM.

Experiment 3

Experiment 1 showed that curvature consistency is a crucial factor in remembering and reproducing motion sequences. The presence of curvature extrema imposed by violations of CW or CCW motion trends in a sequence substantially increased its difficulty. But when does inconsistency take its toll? One possibility is that two (or more) segments moving in a CW/CCW direction act to set an anticipation of upcoming motion in the same direction, so subjects are better at encoding the next segment when it is consistent with the previous curvature, i.e., with their expectation (Ball & Sekuler, 1980, 1981; Sekuler et al., 1990). In fact, anticipation plays an important role in accounts of subjects' segmentation of more complex events (Schwan & Garsoffky, in press). Alternatively, it is possible that everything is in the shape: Subjects' capacity to represent the trajectory could be reduced with more complex geometrical patterns, even if all the individual segments were perceived equally well. The purpose of Experiment 3 was to examine whether the effects of curvature inconsistency were related to the novelty, and therefore uncertainty, of the stimulus model. To this end, we revisited data we had collected earlier to study learning over repeated presentations (see Agam et al., 2007). In that study, subjects viewed and reproduced each stimulus model three consecutive times. If curvature inconsistency impaired performance by way of violating subjects' instantaneous anticipation, then by the second presentation the effects seen in Experiment 1 should be largely absent, as subjects would be familiar with the overall “gist” of the model. 

Methods

Ten normal subjects, none of whom had participated in the previous experiments, ages 18–21 (7 female), took part in this experiment after providing written informed consent. A detailed description of the learning paradigm is provided elsewhere (Agam et al., 2007), so only a brief outline will be given here. The details of the stimulus models and the imitation procedure were the same as Experiments 1 and 2. Here, however, subjects were afforded three opportunities to view and imitate each model. They were given feedback on their performance only following their third (and last) imitation. The feedback took the form of their own three drawn trajectories, color-coded, superimposed on top of a static image of the stimulus trajectory. For data analysis, we split the trials into two data sets in three different ways, exactly as we did with the data shown in Figure 3. Because the data described here were collected for different purposes (Agam et al., 2007), curvature consistency was not explicitly controlled; the only constraints built into the stimulus generator were (i) segments could not intersect, and (ii) inter-segment angles had to lie between 30 and 150 degrees (see Methods). Since such a generation scheme favors curvature inconsistency, the number of trials was not equal for all groups: The “curvature consistent” data sets included about 60 trials per subject, whereas “curvature inconsistent” data sets had approximately 100. 

Results

Figure 5 shows serial position curves for the three stages of learning. Each panel corresponds to two subsets of the data, which differ in curvature consistency at one serial position. Note that following the first presentation of the models (red), the curves exhibit a similar relationship to those in Figure 3. The interesting addition to the previous experiments, however, comes from examining imitation errors following the repetitions of the stimulus. It appears that the consistency-dependent differences in performance are retained, at least through the second imitation (green), and even following the third (blue). Significant interactions between curvature consistency and serial position were found for the second presentation when data were divided according to κ ₂ ( p < 0.002, two-way ANOVA) and for the second ( p < 0.004) and third ( p < 0.05) presentations when κ ₃ was used as a criterion. The persistence of the differences in the curves indicates that the deleterious effects of curvature inconsistency are at least partially resistant to familiarity with the motion sequence. 

Figure 5

 

Experiment 3, learning by repetition. Each panel corresponds to one of the data groups described in Figure 3, as indicated by the gray shading. solid and dashed lines denote consistent and inconsistent models, respectively. The red line represents orientation errors following the initial presentation of a novel model. The green and blue lines indicate the second and third presentations of the same model, respectively. Error bars are within-subject SEM for each curve independently.

Discussion

The experiments reported here investigated factors that affect the ability to encode, remember, and reproduce sequences of directed motions. In Experiment 1, we examined the influence of the angular relationships between segments, namely whether the disc's motion conformed to or violated CW or CCW trends from previous motion segments. We found that the more often the disc toggled between CW and CCW motion, thus creating curvature extrema in its path, the more errors subjects made in reproducing its trajectory. In Experiment 2, we tested whether the outcome of Experiment 1 may have been due to different spatial properties of the stimulus models indirectly imposed by the angular constraints. We generated groups of stimuli that occupied spatial regions of a different size. This manipulation, however, showed no effect on performance; Subjects reproduced models from each group with equivalent accuracy. Experiment 3 showed that differences in performance due to curvature inconsistency persist even after the stimulus model has become familiar. 

In light of our results, it seems that the typical bow-shaped serial position curves we reported previously (Agam et al., 2005, 2007) could in fact be an average of many types of functions. This may be true for other serial recall studies (e.g., Madigan, 1980), where internal list structure might make some lists easier to encode and remember than others. 

Some caution is needed when interpreting the results of Experiments 1 and 2, as the variables we manipulated can indirectly affect other, correlated variables. Most notably, introducing regularities in the direction of motion (CW or CCW) affects the angles between pairs of adjacent segments. This occurs because our stimulus generation rules did not allow motion segments to intersect (see Methods). Stimuli of type +++, for example, tend to comprise one or more oblique angles, or else the disc's motion would form a forbidden, closed shape. To determine whether this correlated variable could explain the observed differences in behavioral performance, we calculated the relative angles in all the stimulus models used in the Experiment 1. Figure 6A shows the angles between adjacent segments at the four possible positions, for each one of the stimulus subgroups described in Figure 3. The angles between segments are indeed more oblique in regions where curvature is consistent, indicating that relative angle differences could be a possible source of heterogeneity. A different picture, however, appears in Experiment 2. Figure 6B shows the relative angles for stimulus models in the three distance conditions of Experiment 2. Clearly, there is a large difference in the distribution of these angles between experimental conditions (much more so than in Experiment 1), yet there was no difference whatsoever in imitation accuracy ( Figure 4). This suggests, then, that angular differences between segments are insufficient to explain the main result of Experiment 1, strengthening the idea that curvature consistency in the disc's motion path is the main factor underlying the shape of the serial position curve. 

Figure 6

 

Angles between segments in stimulus models. (A) Experiment 1. Stimuli are grouped the same way as in Figure 3, as indicated by the gray shading. (B) Experiment 2. Each data point corresponds to the angle between two segments at serial positions given by the two numbers separated by a dash on the abscissa. Note that these are not behavioral results, but properties of the stimulus models.

Experiment 3 showed that the effect of curvature extrema largely remains when the stimulus model is familiar to the subject (at least in the course of three presentations). This suggests that certain trajectories may be inherently more difficult to encode and reproduce, even when the general, overall structure of the model has become known. But why would particular trajectory shapes be harder to encode than others? As we noted earlier, the answer may come from the object and shape representation literature. It has long been argued that most of the information along a contour lies in points of high curvature (Attneave, 1954; Resnikoff, 1985). This assertion has since been extended to distinguish between convexities (positive curvatures) and concavities (negative curvatures), defined with respect to the shape's interior (Braunstein, Hoffman, & Saidpour, 1989; Feldman & Singh, 2005; Hoffman & Richards, 1984; Richards et al., 1986). Negative curvatures, in particular, have been thought of as a significant carrier of information, defining how a shape is parsed by the visual system. Hoffman and Richards (1984) put this on formal ground with their “minima rule,” which states that human vision divides a shape into parts at negative minima of curvature or points of maximum concavity. This influential theory is consistent with psychophysical findings related to perception and memory for static shapes (e.g., Barenholtz et al., 2003; Braunstein et al., 1989; Hoffman & Singh, 1997). 

Our motion sequences comprise sets of discrete linear components and therefore do not form continuous contours. Nor do they constitute static shapes in the usual sense of the term. Instead, the trajectories are produced by one visible point whose position changes systematically over time. Yet, it is conceivable that when subjects construct a mental representation of the trajectories, the information is organized according to rules that are similar to ones that would be used for static shapes, namely, using negative curvatures as part boundaries. In fact, Shipley et al. (2004) found that subjects segmented the motion path of a dot in a very similar way to how they segmented static objects, which in turn was consistent with the minima rule. We should then ask ourselves how a negative curvature is to be defined in the absence of a closed shape (remember that our models were forced to construct open-ended patterns). We propose that as long as the disc moves continuously CW or CCW, it can be thought of as moving along the edge of a shape, whose interior lies in the part of space enclosed by the disc's path, were it to proceed in the same CW or CCW direction. Once the disc changes its course from CW to CCW or vice versa, a curvature extremum is present, and the subject is “forced” to create a new shape entity in his or her mind, thereby increasing the amount of information that must be kept in memory for successful imitation. In other words, continuous curvature facilitates integration of component segments into a single unit in memory because they can be represented as a single shape; curvature inconsistency calls for a multiple-part representation. As visual working memory is extremely limited in capacity (Olsson & Poom, 2005; Todd & Marois, 2004; Vogel & Machizawa, 2004; Vogel, Woodman, & Luck, 2001), it should not be surprising that even a single partition entails a cost in imitation performance. This hypothesis is supported by the fact that imitation error increases around the position where the presumed partition took place, namely, the point at which curvature becomes inconsistent (Figure 3). 

Feldman and Singh (2005) suggested a compact measure for the complexity of a shape, based on information theory (Shannon, 1948): The integral of log (p), with p being the probability of a change in the tangent direction on a smooth curve. This “surprisal” measure, in turn, is proportional to the negative cosine of the angle created by successive points along the contour. This way, curvature extrema maximize shape information. Particularly, negative extrema carry the most information, as they are less likely to occur in a closed shape than positive extrema. Feldman and Singh point, then, to two factors governing the information content along a contour: the magnitude of the turning angle and its sign. In our case, angle magnitude cannot account for the differences in performance between stimulus types, as discussed earlier in this section. However, the robust effect of the sign of the turning angle may be a direct consequence of the subjects' treatment the motion path as defining a shape, with inconsistent turns leading to higher “surprisal” and more information to be encoded. Integrating over these negative turning points should then lead to increased error, which is exactly the pattern seen in Figure 2B. 

Figure 7 shows some examples of how a chunking mechanisms could work for different stimulus configurations. Four examples from Figure 1B are used to illustrate chunking along trajectories. Note that there may be other possible ways to partition each model. For example, the second segment in the −++ model could belong to the first, smaller chunk rather than to the second chunk. (Even so, that particular model would still consist of two parts.) This representational scheme could explain the shape of the imitation error curves in Figure 2A. For instance, if an entire +++ model were encoded as a single chunk ( Figure 7), then it would not be surprising that the corresponding serial position curve ( Figure 2A, top left) reveals little difference between individual segments. To further illustrate this point, consider a −++ model and its corresponding error curve. The imitation error is smallest for the first segment, and then it increases and remains relatively uniform across the rest of the model ( Figure 2A, bottom left). This is consistent with chunking of segments 2 to 5 into a single representation, as demonstrated in Figure 7. 

Figure 7

 

Chunking of motion sequences. Each panel shows a model of a different type, taken from Figure 1. The small, dotted circles represent the starting point of the trajectory. The dashed ellipses enclose parts of the sequence that are chunked together and represented as one part. Note that each of these examples is not the only possible transformation for a given model.

How exactly sequences are chunked needs to be further investigated. As we mentioned, curvature inconsistency does not lead to an unambiguous division, as a segment that violates the trajectory's consistency may serve as the final segment of the current chunk or as the first segment of the next. A Bayesian approach may be useful here, with an explicit definition of the probability of violating a “single chunk” hypothesis at every serial position in the sequence. A Bayesian model successfully accounts for integration of static dots into a contour (Feldman, 2001). It might then be possible to extend it to the integration of contour segments into a virtual shape. The main factor, of course, influencing the probability of creating a new part would be the sign of the turns. However, additional factors, such as the magnitude of the turning angle, might influence which part a segment at a negative extremum would belong to. 

In conclusion, we propose that curvature inconsistencies promote the partitioning of motion sequences into multiple parts. Such partitioning may increase memory load and consequently impair performance, as effectively more components need to be remembered. Another way to think about this is that recoding of the sequences into chunks is more difficult (or less efficient) in the presence of curvature extrema. Would this assertion hold true for any number of sequence components? Probably not. As sequences grow longer, representing a large number of segments as a single entity becomes less practical. In that case, dividing the trajectory into multiple parts could be beneficial. 

Acknowledgments

Supported by the National Science Foundation grant SBE-0354378 (Center for Learning, Education, Science & Technology). 

Commercial relationships: none. 

Corresponding author: Yigal Agam. 

Email: [e-mail]. 

Address: 300 Longwood Ave., Boston, MA 02115.