On the effective number of tracked trajectories in normal human vision

Abstract

Z. W. Pylyshyn and R. W. Storm (1988) have shown that human observers can accurately track four to five items at a time. However, when a threshold paradigm is used, observers are unable to track more than a single trajectory accurately (S. P. Tripathy & B. T. Barrett, 2004). This difference between the two studies is examined systematically using substantially suprathreshold stimuli. The stimuli consisted of one (Experiment 1) or more (Experiments 2 and 3) bilinear target trajectories embedded among several linear distractor trajectories. The target trajectories deviated clockwise (CW) or counterclockwise (CCW) (by 19°, 38°, or 76° in Experiments 1 and 2 and by 19°, 38°, or 57° in Experiment 3), and observers reported the direction of deviation. From the percentage of correct responses, the “effective” number of tracked trajectories was estimated for each experimental condition. The total number of trajectories in the stimulus and the number of deviating trajectories had only a small effect on the effective number of tracked trajectories; the effective number tracked was primarily influenced by the angle of deviation of the targets and ranged from four to five trajectories for a ±76° deviation to only one to two trajectories for a ±19° deviation, regardless of whether the different magnitudes of deviation were blocked (Experiment 2) or interleaved (Experiment 3). Simple hypotheses based on “averaging of orientations,” “preallocation of resources,” or pop-out, crowding, or masking of the target trajectories are unlikely to explain the relationship between the effective number tracked and the angle of deviation of the target trajectories. This study reconciles the difference between the studies cited above in terms of the number of trajectories that can be tracked at a time.

Introduction

How many moving items can we track simultaneously? This question has attracted the attention of the scientific community for a long time, and the last two decades have seen focused interest on this question under the name multiple object tracking (MOT). Pylyshyn and Storm (1988) showed that human observers have the ability to track up to four or five items simultaneously with a high degree of reliability. A number of MOT studies have used variations of the Pylyshyn and Storm paradigm and found the number of items that can be tracked simultaneously to be comparable to that found in the original study (Bahrami, 2003; Oksama & Hyona, 2004; Scholl & Pylyshyn, 1999; Scholl, Pylyshyn, & Franconeri, 1999; Viswanathan & Mingolla, 2002; Yantis, 1992). It is now accepted that practiced human observers can, under a variety of circumstances, track up to four or five objects reliably. 

A recent study examined how the threshold for detecting a deviation in a straight-line trajectory changed as the number of trajectories was varied (Tripathy & Barrett, 2004). In this study, the target followed a bilinear trajectory, with a single deviation at the midpoint of the trajectory, whereas the distractor trajectories followed straight-line paths. The angle of deviation of the target trajectory was varied and observers reported the observed direction of deviation [clockwise (CW) or counterclockwise (CCW)]. From the observer's responses, the threshold for detecting deviations in trajectories was determined. When the target was the only trajectory present, thresholds for detecting deviations were typically around 2°. However, when the number of trajectories was increased (by adding linear, distractor trajectories) thresholds increased rapidly; adding a single distractor elevated the threshold almost by a factor of four relative to that for the target presented alone. Thus, in this threshold-measuring paradigm, observers were unable to effectively track more than one trajectory at a time. 

There are several differences between the Pylyshyn and Storm (1988) paradigm and that used in Tripathy and Barrett (2004), and these have been described by Tripathy and Barrett. Any of these differences, either individually or combined, could have been responsible for the inability of the observers in the latter study to track more than a single trajectory at a time. One obvious factor that could account for the difference between the results of the two studies might be the difficulty of the tasks involved. In the Tripathy and Barrett study, because the objective was to measure deviation thresholds, the angles of deviation of the target trajectory were small, typically in the neighborhood of the observer's deviation threshold. It is possible that had the angles of deviation of the target trajectory been greater, observers would have been able to track more than a single trajectory at a time. Thus, in order to understand the difference in the results of the two studies, we measured the performance of observers when tracking deviations in multiple trajectories when the angles of deviation were substantially suprathreshold, that is, much larger than the threshold deviation of the target trajectory presented by itself (the actual deviations used in most experiments were ±19°, ±38°, and ±76°). From each observer's performance, we estimated the “effective” number (see Methods section) of tracked trajectories for a wide range of experimental conditions. 

In the first experiment, there was one target trajectory and the number of distractor trajectories was varied. In the second experiment, the paradigm was generalized to include varying numbers of target trajectories and distractor trajectories; the number of target and distractor trajectories and the angle of deviation were fixed within a block but could vary between blocks. In each case, the effective number of tracked trajectories was estimated. Our main finding is that, over the range of parameters tested, the effective number of tracked trajectories is largely uninfluenced by the number of target trajectories or the number of distractor trajectories but is primarily influenced by the angle of deviation of the target trajectory (or trajectories). The third experiment was a repetition of the second experiment, but with the different numbers of target trajectories and distractor trajectories and the different angles of deviation all interleaved within a block. We had anticipated that without prior knowledge of the difficulty of the task the observer would be unable to predetermine the amount of resources to allocate to each trajectory, and this would result in the effective number of trajectories tracked being qualitatively different from those of the second experiment. Surprisingly, this turned out not to be the case—the effective numbers of trajectories tracked when the experimental conditions were interleaved were qualitatively and quantitatively similar to those when the experimental conditions were blocked and again were primarily influenced by the angle of deviation of the target trajectories. 

Methods

Equipment

The stimuli were generated on a Gateway 2000 computer and displayed on a Vivitron 1776 monitor. The monitor had a frequency of 60 Hz, yielding a frame duration of 16.67 ms. The portion of the monitor screen that was used for displaying the stimulus was 798 × 574 pixels. The viewing distance was 1.29 m. At this distance, each pixel subtended 1′ in the horizontal and vertical directions, and the corresponding dimensions of the monitor screen were approximately 13.3° and 9.6°, respectively. Chin and forehead rests were used to minimize head movements during the experiment. Experiments were conducted with the room lit normally using two fluorescent lights, thus ensuring that observers could not utilize the persistence of the trajectories on the screen for making their responses. 

Stimulus and procedure

The stimuli employed are similar in nature to those described in Tripathy and Barrett (2004) (Figures 1 and 2). One or more dots (dot luminance 55.4 cd/m², background luminance 2.4 cd/m², angular subtense 5′ × 5′) underwent apparent motion across the computer screen in a left-to-right direction. The dots moved with a speed of 4 deg/s along either a linear or a bilinear trajectory (i.e., a trajectory with a single deviation in direction), and the entire trajectory presentation lasted for a total of 51 frames (850 ms). Two vertical markers indicated the midline of the screen, and observers were aware that any deviation that may arise in a trajectory would take place when the dot was exactly halfway along its trajectory, that is, between the vertical markers (Figures 1 and 2). 

Figure 1a, 1b

 

Schematic of typical stimuli used in Experiment 1. The stimuli are not drawn to scale. The solid lines schematically represent the trajectories of the dots during the course of a hypothetical trial (on any frame during the motion, only a single dot per trajectory was present on the screen, yielding a stimulus that was substantially less cluttered than shown here). For the stimuli shown here, the overall number of trajectories (T) is three in (a) and six in (b). All trajectories moved from left to right on the screen. Halfway along its trajectory (indicated by the white, vertical markers), one of the dots underwent a deviation (CW or CCW) the magnitude of which was fixed within a block (−76° in panel a and +76° in panel b) and could vary between blocks. The deviations within a block were all ±19°, all ±38°, or all ±76°. All dots traveled at a speed of 4 deg/s.

Figure 2a, 2b

 

Schematic of typical stimuli used in Experiment 2. As in Figure 1, the solid lines schematically represent the trajectories of the moving dots. Stimuli with 10 trajectories are shown ( T = 10). Unlike the previous figure, the number of deviating trajectories ( D) could be greater than one, D being three in panel a and six in panel b. The magnitude of deviation (±76° deviation is shown) and D were fixed within a block and could vary between blocks. All deviations on a trial were of the same magnitude and direction. Other details are as in Figure 1.

We conducted three experiments in which different numbers of deviating (bilinear) and undeviating (linear) trajectories were presented. In the first experiment, only one deviating trajectory (target trajectory) was presented, and the number of nondeviating trajectories (distractors) was fixed within a block of trials but was varied between blocks. In the second experiment, the overall number of trajectories presented remained constant, and the number of deviating trajectories (targets) and the magnitude of their deviation were fixed within a block but was varied between blocks. The psychophysical procedures employed in the above two experiments were identical. Specifically, within a block of trials, the magnitude of the deviations that occurred was fixed, and the deviations were either CW or CCW. On a trial with more than one target trajectory, all of the target trajectories deviated in the same direction and had the same angle of deviation. On half of the trials in a block, the trajectories had CW deviations. Following each trial, the observer reported the direction of the perceived deviation, CW or CCW, by pressing the appropriate key on the keyboard. Feedback as to the correctness of the observer's response was provided in the form of one of two audible tones. The results were expressed as the percentage of correct identifications of the direction of deviation for the particular combination of the angle of deviation, the overall number of trajectories, and the number of deviating trajectories. The third experiment was similar to the second experiment, but the numbers of target trajectories and distractor trajectories and the different angles of deviation of the targets were all interleaved within each block. As in the second experiment, the results were expressed as the percentage of correct identifications of the direction of deviation for the different experimental conditions. 

The selection of dot speed of 4 deg/s was based on our previous findings that thresholds for detecting deviations in similar tasks were lowest when speeds between 2 and 8 deg/s were employed (Experiment 4 of Tripathy & Barrett, 2004). The trajectories were randomly oriented within ±80° from the horizontal (0°). The random orientations of the trajectories ensured that the trajectories were not parallel. Thus, deviation from parallelism could not be used as a cue for identifying the direction of deviation. When more than one trajectory was present on a trial, the different trajectories were permitted to intersect. However, the starting points and orientations of the trajectories were selected so that the dots constituting the trajectories were exactly aligned vertically at the midline of the monitor on the 26th frame (when the deviation(s) occurred). On this 26th frame, the average vertical interdot separation was 40′ (±5′) in the first experiment and 30′ (±5′) in the second and third experiments. This intertrajectory separation at the instant of deviation(s) ensured that intersections among the trajectories did not mask their deviation(s). 

Experiment 1: 1-of-T trajectories deviating

The stimulus used in this experiment is schematically represented in Figure 1. The stimulus on each trial consisted of a number of trajectories (three in Figure 1a and six in Figure 1b), of which only one deviated. Within each block of 100 trials, the magnitude of deviation of the target trajectory was fixed (the direction of deviation on a trial could be either CW or CCW), as was the total number of trajectories ( T) presented on each trial. For each combination of deviation angle and T, observers completed a practice block followed by two further blocks, and the percentage of correct responses was determined from these 200 trials. For each of the three deviations employed (± 19°, ± 38°, and ± 76°), data were gathered for values of T = 1, 2, 3, 4, 6, 8, and 10 trajectories. The number of distractor trajectories present was always one less than T. 

Experiment 2: D-of-T trajectories deviating: conditions blocked

The stimulus used in this experiment is schematically represented in Figure 2. In this case, the total number of trajectories ( T) was held constant and the number of deviating trajectories ( D) was varied. The number of distractor trajectories was therefore given by ( T − D). As in Experiment 1, the magnitude of the deviation angle within a block of trials was fixed, as were D and T. Observers knew in advance how many trajectories ( T) would be presented in the block of trials, how many of these would deviate ( D), and the magnitude of the deviation. Data were collected for T = 10 for deviations of ±19°, ±38°, and ±76° for values of D = 1, 2, 3, 5, 7, and 10. As in the previous experiment, the percentage of correct responses for each combination of deviation angle and D was determined from two blocks yielding 200 trials. The experiment was also conducted for the same deviation angles when the total number of trajectories was eight (i.e., T = 8, with D = 1, 2, 3, 5, 6, and 8) and six (i.e., T = 6, with D = 1, 2, 3, 4, and 6). 

Experiment 3: D-of-T trajectories deviating: conditions interleaved

The stimulus used in this experiment was similar to that in the previous one; however, the different experimental conditions were interleaved. The total number of trajectories ( T) was held constant and the number of deviating trajectories ( D) was varied from trial to trial, as was the magnitude of the deviation angle. Observers knew in advance how many trajectories ( T) would be presented in the block of trials, but did not know how many of these would deviate ( D), or the magnitude of the deviation. Only a restricted range of values of D was used and the range of deviation angles was also reduced; this was done to reduce any potential cues from the average orientation of the trajectories (see Results and discussion section, Experiment 2). One set of data was collected for T = 10 with deviations of ±19°, ±38°, and ±57° for values of D = 1 and 2 (the six combinations from three orientations and the two values of D were interleaved). Another set of data was collected for T = 10 with deviations of ±19°, ±38°, and ±57° when D = 1, with deviations of ±19° and ±38° when D = 2, and with deviations of ±19° when D = 3. In this experiment, each block consisted of 120 trials, with 20 trials for each of the six combinations of deviation angle and the value of D interleaved within a block. The percent correct responses for each combination of deviation angle and D was derived from 15 blocks yielding 300 trials for each stimulus condition (which was larger than the 200 trials for each stimulus condition in the previous experiment). 

Observers

The three authors served as observers. Authors SS and ST participated in all experiments while BB participated in most experiments. All three had normal or corrected-to-normal vision and performed the experiment binocularly. All three were experienced psychophysical observers and, at the time of doing these experiments, had substantial experience as observers in MOT studies. Observers were not required to fixate any particular point on the screen and were permitted to pursue the trajectories, if they subjectively felt that it made the task easier. 

The predicted performance ( P) of a limited capacity hypothetical observer

Consider a hypothetical observer that has a limited amount of attentional and tracking resources available and distributes these resources among the maximum number of trajectories that it can track perfectly; consider this to be A trajectories. If the capacity of this observer was large, then A would be large. If this hypothetical observer was presented with a stimulus containing fewer than A trajectories, then its performance for identifying the direction of deviation of the target trajectories would be perfect. However, if it was presented with stimuli containing more than A trajectories, then its performance would be less than perfect. 

What will be the performance of this limited capacity hypothetical observer (LCHO) that perfectly tracks A randomly selected trajectories out of T and ignores the remaining ( T − A) trajectories? To determine the performance of this LCHO, we assumed that the trajectory deviations are much larger than its deviation threshold for a single trajectory; that is, the deviation on a trial will be detected correctly if the deviating trajectory is among the A trajectories allocated adequate computational resources. 

In the case of Experiment 1, when the deviating trajectory is among the ( T − A) trajectories that are not allocated computational resources, the LCHO has inadequate information regarding the direction of deviation and must select the direction of deviation at random. All available computational resources are directed to only the A randomly selected trajectories. The percentage of correct responses P can be determined from the following equations from basic probability:  

$P=100\mathrm{f}\mathrm{o}\mathrm{r}(A\ge T),$

(1)

 

$P=100\times [A/T+(1-A/T)/2]\mathrm{f}\mathrm{o}\mathrm{r}(A<T).$

(2)

 

Equation 1 states that if each trajectory is allocated adequate resources, then the LCHO will always be correct. In Equation 2, the first term in the rectangular parenthesis ( A/ T) is the probability that the deviating target is among the A trajectories that are allocated adequate computational resources (i.e., the probability of knowing the direction of deviation), whereas the second term ((1 − A/ T) / 2) is the probability of correctly guessing the direction of deviation. Based upon these assumptions, we have generated predicted performance levels for Experiment 1 for different numbers of trajectories effectively tracked ( A) by the LCHO. These predicted performance levels are shown in Figure 3. For example, if six trajectories are presented ( T = 6) and the LCHO can process only three trajectories ( A = 3) effectively, the probability that a correct response will be made is 0.75 [i.e., 100 × (3/6 + (1 − 3/6) / 2), or 75% correct in Figure 3]. 

Figure 3

 

Predictions for the LCHO when one out of T trajectories deviates. Predictions (in percentage of correct responses) in the binary-choice task (CW or CCW) employed in Experiment 1 for the different numbers of trajectories ( T) in the stimulus. These predictions represent the expected performance levels of an LCHO that perfectly tracks A of the T trajectories presented and have been generated using Equations 1 and 2. Predictions are shown for different values of A. For example, if the LCHO can perfectly track six trajectories ( A = 6), performance will not drop below 100% correct unless the total number of trajectories presented exceeds six. Actual performance levels of human observers can be converted into effective numbers of tracked trajectories using these predicted performance levels.

Similar performance predictions can be made for the LCHO under the conditions of Experiment 2 (and also Experiment 3). The parameters of interest are the number of deviating trajectories ( D), the maximum number of trajectories that the LCHO can track perfectly ( A), and the total number of trajectories ( T). In this case, the assumption made is that the direction of deviation will be correctly identified if any of the D deviating trajectories are among the A trajectories that are tracked by the LCHO. When all the D deviating trajectories are among the ( T − A) trajectories that are not allocated computational resources, the LCHO has no information regarding the direction of deviation and must determine the direction randomly. The percentage of correct responses P can be determined similarly from simple probability using the following equations:  

$P=100\mathrm{f}\mathrm{o}\mathrm{r}(D>(T-A\left)\right),$

(3)

 

$P=100\times [(1-\left(\begin{array}{c}T-D\\ A\end{array}\right)/\left(\begin{array}{c}T\\ A\end{array}\right))+(\left(\begin{array}{c}T-D\\ A\end{array}\right)/\left(\begin{array}{c}T\\ A\end{array}\right))/2]\mathrm{f}\mathrm{o}\mathrm{r}(D\le (T-A)).$

(4)

 

Equation 3 states that if D > ( T − A), then the A trajectories that are allocated computational resources must contain at least one deviating trajectory. In Equation 4, the first term in the rectangular parenthesis represents the probability that there will be at least one deviating trajectory among the A trajectories allocated computational resources (i.e., the probability of knowing the direction of deviation). The second term is the probability of correctly guessing the direction of deviation; the LCHO must resort to guessing when there is no deviating trajectory among the A trajectories that are allocated resources. 

Figure 4 plots the predicted performance levels for Experiments 2 and 3 as the number of deviating trajectories is varied, keeping the total number of trajectories ( T) fixed at 10. The predictions are shown for different numbers of maximum trajectories tracked ( A) by the LCHO. For example, if 10 trajectories are presented ( T = 10), of which 5 deviate ( D = 5), but the LCHO can track only 3 trajectories ( A), the probability of a correct response is 0.96 [i.e., 100 × ((1 − 10 / 120) − (10 / 120) / 2), or 96% correct in Figure 4]. The percent correct curves in Figure 4 are shown as continuous, whereas Equation 4 is defined only for integer values of D; this has been done in order to facilitate the visualization of the functions. Similar predictions can be generated when T = 8 and T = 6. 

Figure 4

 

Predictions for the LCHO when D out of 10 trajectories deviate. Predictions (in percentage of correct responses) in the binary-choice task employed in Experiment 2, for the different numbers of deviating trajectories ( D) in the stimulus, when there are a total of 10 trajectories ( T = 10). These predictions represent the performance of an LCHO that perfectly tracks A out of the 10 trajectories presented and are derived from Equations 3 and 4. The predictions are shown for different values of A. For example, if three trajectories can be tracked perfectly, a 75% correct performance is predicted when 2 of the 10 trajectories deviate (i.e., when A = 3, D = 2, and T = 10). Actual performance levels of human observers can be converted into effective numbers of tracked trajectories using these predicted performance levels.

Equations 1 and 2 are algebraically similar to expressions derived for the expected level of performance in Scholl, Pylyshyn, and Feldman (2001, see their Appendix). Hulleman (2005) has suggested some refinements to these equations; but as Hulleman noted, these refinements apply to neither the paradigm of Tripathy and Barrett (2004) nor the current experimental paradigm. 

The LCHO proposed here might not be using an ideal strategy. It is an open question whether a hypothetical observer that selectively tracks A trajectories perfectly, like our LCHO does, will perform better than another hypothetical observer that distributes its resources (attentional, computational) so that it tracks say A + 2 trajectories slightly less than perfectly. 

The effective number of tracked trajectories ( E)

We define the human observer's effective number of tracked (attended?) trajectories under a set of experimental conditions to be E, if the performance of the human observer under those conditions is comparable to that of the LCHO with A = E. In other words, E is the specific value of A for which the performance of the LCHO matches that of the human observer under a set of experimental conditions. To determine E, we need to compare the human observer's performance to those of the LCHO for different values of A and find the value of A that yields the closest match. One could interpret the effective number tracked to be the number of trajectories that are assigned adequate computational resources to be able to detect a deviation of the magnitude used in the experimental condition. However, we do not wish to imply that the human observer perfectly tracks E trajectories and ignores the remaining ( T − E) trajectories. We only imply that the human observer's performance is comparable to that of a hypothetical observer that perfectly tracks E trajectories and ignores the remaining ( T − E) trajectories. For example, if the human observer's performance is comparable to a hypothetical observer that tracks E trajectories perfectly, it could also be the case that the observer's performance is comparable to another hypothetical observer that tracks, say ( E + 2) trajectories but less than perfectly. In general, the effective number of tracked trajectories should be seen as a description of human performance and not as an explanation of human performance. However, if under a variety of conditions the estimated effective number of tracked trajectories remains roughly constant, then a parsimonious explanation for this is that this estimate is a reflection of the actual number of trajectories tracked; in other words, the observers could really be tracking E trajectories and ignoring ( T − E) trajectories, and E might represent some internal capacity limit for tracking for this observer in the experimental condition being tested. 

Estimating the effective number of tracked trajectories ( E) from human performance

We also used a graphical technique to estimate the effective number of tracked trajectories from the human observers' percentage of correct responses. This technique is described here for Experiments 2 and 3; Experiment 1 is a special case of Experiment 2 with D = 1. Let us consider the case when there are 10 trajectories, with 2 trajectories deviating (i.e., T = 10; D = 2). The predicted performance for the LCHO that tracks A trajectories perfectly can be determined for A = 1, 2,…, 10 by taking a vertical slice through Figure 4 at D = 2. The expected percent correct when A = 0 is 50% and when A = 10 is 100%. These data have been replotted in Figure 5a with the number of trajectories tracked by the LCHO along the abscissa and then fitted with an exponential curve. This enables the human observer's percentage of correct responses to be graphically mapped on to its corresponding number of tracked trajectories as shown in Figure 5a. This will yield the effective number of tracked trajectories ( E) when there are 10 trajectories in the stimulus with 2 deviating ( D). If two observers had performance levels of 84% and 80.5% correct responses under these experimental conditions (these percentages correspond to actual data taken from Experiment 2 for a 76° deviation, see Figure 7a, red symbols at D = 2), then their corresponding numbers of tracked trajectories will be 4.10 and 3.48, respectively, as can be seen in Figure 5a. In other words, for the experimental condition described here, the performance of the first observer was comparable to that of the LCHO when it tracked 4.10 trajectories perfectly, and that for the second observer yielded 3.48 for the effective number of tracked trajectories. Note that the fractional numbers obtained here for the effective numbers of trajectories tracked perfectly are not counterintuitive. One can readily imagine a hypothetical observer that tracks four trajectories on most trials and tracks five trajectories on a few trials to yield an average of 4.10 trajectories tracked perfectly. 

Figure 5a, 5b

 

Estimating the effective number of tracked trajectories. (a) The symbols shown replot the data from Figure 4 for the condition with two deviating trajectories ( D = 2). The predicted percent correct performance is plotted for different numbers of tracked trajectories ( A), for A = 0, 1, 2, 3,…, 10. The dotted line is an exponential fit to the data. Given the observers' performance, a direct estimate of the effective number of tracked trajectories ( E) can be obtained. For example, if a human observer scores 80.5% correct when 2 out of 10 trajectories deviate, this performance is consistent with 3.48 trajectories being tracked perfectly. A performance of 84% correct corresponds to 4.10 trajectories being tracked perfectly. (b) The entire family of curves showing the predicted performance for different numbers of tracked trajectories, for different values of D, when the stimulus consists of 10 trajectories.

Figure 5b shows the different curves obtained by taking vertical slices at different abscissa values in Figure 4, with each curve corresponding to a particular value of D (in the case of D = 1, a straight-line fit was used instead of an exponential fit). For any performance level of the human observer obtained with T = 10, the number of tracked trajectories can be estimated by projecting the observed performance level to the curve corresponding to the number of deviating trajectories ( D) and reading off the appropriate abscissa value. Similar figures (not shown) were constructed for T = 8 and for T = 6, and these were used for estimating the effective number of tracked trajectories under these conditions. This graphical technique yielded more accurate estimates for the effective number of tracked trajectories compared to estimates obtained visually from Figure 4. 

Results and discussion

Experiment 1: 1-of- T trajectories deviating

Figure 6 shows how the percentages of correct responses of the three human observers change with the total number of trajectories ( T) when there was only one trajectory deviating. Also shown (dotted lines) are the predictions for the performance of the LCHO that perfectly tracks one, two, three, or four trajectories. These predictions are a subset of the curves shown in Figure 3. 

Figure 6

 

Results of Experiment 1. The percentage of correct responses of the observers is plotted as a function of the number of trajectories ( T) in the stimulus. The different symbols identify the three observers. The different colors green, blue, and red identify the data for the different angles of deviation tested (±19°, ±38°, and ±76°, respectively). The dotted lines represent the predictions of performance when different numbers of trajectories are tracked perfectly ( A = 1, 2, 3, 4). The performance of the observers drops as T increases for all three angles of deviation. Also evident is the substantial improvement in performance as the magnitude of the angle of deviation is increased.

Across the range of the number of trajectories ( T) tested, the effective number of tracked trajectories ( E) was between one and two when the angle of deviation was ±19°, between two and three when the angle of deviation was ±38°, and between three and four when the angle of deviation was ±76°. The next experiment investigated the effective number of tracked trajectories more systematically. 

Experiment 2: D-of- T trajectories deviating: conditions blocked

In the second experiment, the number of trajectories was 10 initially and the number of deviating trajectories varied. Figure 7a shows the percent correct responses against the number of deviating trajectories. Data are shown for the three observers. Also shown (dotted lines) are the predictions for the performance level of the LCHO that tracks A trajectories, where A = 1, 2,…, 5 and ignores the remaining trajectories. These predicted performance levels are a subset of the curves shown in Figure 4. The results in Figure 7a seem comparable to those in Experiment 1, with observers effectively tracking one to two trajectories when the deviation is ±19°, two to three trajectories when the deviation is ±38°, and three to five trajectories when the deviation is ±76°. 

Figure 7a, 7b, 7c, 7d, 7e, 7f

 

Results of Experiment 2. (a) Observers' performance (in percentage of correct responses) is shown as a function of the number of deviating trajectories ( D) when the stimulus consisted of 10 trajectories ( T = 10). The different colors represent the different angles of deviation (±19°, ±38°, and ±76°) of the target trajectories and the different symbols represent the different observers. Predicted performance curves for an LCHO that perfectly tracks one to five trajectories ( A = 1, 2,…, 5) are shown as dotted lines. (b) Most of the data from (a) have been converted into estimated effective numbers of tracked trajectories ( E) as illustrated in Figure 5, and these have been replotted as a function of the number of deviating trajectories ( D). Data in panel a that corresponded to performance levels that were very close to 100% correct responses or very close to chance (50%) were not plotted in panel b (see text). The solid symbols represent stimulus conditions for which extraneous cues may have assisted performance as indicated by control experiments described under Experiment 2. Data for the three observers have been fitted with a straight line for each of the three deviation angles. Panels c and d show the results obtained with stimuli having eight trajectories ( T = 8), and panels e and f show the results obtained with stimuli having six trajectories ( T = 6). The format of panels c and e is similar to panel a, and the format of panels d and f is similar to panel b. The effective numbers of tracked trajectories ( E) are about one to two trajectories for deviations of ±19°, two to three trajectories for deviations of ±38°, and four trajectories for deviations of ±76°.

Each data point in Figure 7a was converted to an effective number of tracked trajectories as described in the Methods section and then plotted in Figure 7b. Data that were too close to saturation (>96% correct responses averaged over the three observers) or too close to chance (<54% correct responses averaged over the three observers) were not converted to effective numbers of tracked trajectories. Data that were close to saturation were compatible with a range of values for the number of tracked trajectories (notice the flatness of the curves at the upper end in Figure 5b). Data that were below the 50% level (due to random fluctuations in human performance) could not be converted into effective numbers of tracked trajectories. The difference between the open and closed symbols is described later in this section. Straight lines were fitted to the ±19°, ±38°, and ±76° deviation data and these are shown as green, blue, and red dotted lines, respectively. The straight-line fits in Figure 7b show how the effective number of tracked trajectories (averaged over the three observers) changes as the number of deviating trajectories is varied. For deviations of ±19° and ±38°, changing the number of deviating trajectories has only a small effect on the effective number of tracked trajectories. For example, for a ±19° deviation, changing D from 2 to 7 resulted in the effective number of tracked trajectories changing from 1.57 to about 1.19. A 10-trajectory stimulus with 2 trajectories deviating is very different from one with 7 trajectories deviating. However, for both of these very different stimuli, the effective number of tracked trajectories is about 1.38. Likewise, for the ±38° deviation data, a 10-trajectory stimulus with 1 trajectory deviating is very different from a 10-trajectory stimulus with 7 trajectories deviating; however, the effective number of tracked trajectories (averaged over the three observers) in the two cases is 2.74 and 1.96. In the case of the ±76° deviation data, only a limited number of these could be converted to effective numbers of tracked trajectories because performance was close to 100% correct when the number of deviating trajectories was five or more. 

Figures 7c and 7d show, using the same format as above, the data when there were eight trajectories in total. Figure 7c shows the raw data and Figure 7d shows the effective number of tracked trajectories for the three magnitudes of deviation. The effective number of tracked trajectories shown in Figure 7d is both qualitatively and quantitatively similar to that shown in Figure 7b. Observers were, on average, effectively tracking between 1.41 and 1.11 trajectories when the deviation was ±19°, and between 2.36 and 2.08 trajectories when the deviation was ±38°. As before, only a small proportion of the data collected when the deviation was ±76° could be converted into effective numbers of tracked trajectories because of saturation of the performance level. 

Figures 7e and 7f show the data when there were six trajectories in total. The effective numbers of tracked trajectories shown in Figure 7f are similar to the corresponding numbers in Figures 7b and 7d. Observers were, on average, effectively tracking between 1.35 and 1.19 trajectories when the deviation was ±19°, and between 2.05 and 2.56 trajectories when the deviation was ±38°. The corresponding effective numbers of tracked trajectories when the magnitude of deviation was ±76° were between 3.40 and 3.82. However, for all three magnitudes of deviation, only a limited set of effective numbers of tracked trajectories was obtained because the total number of trajectories was only six. 

The best-fitting straight lines to the effective numbers of tracked trajectories from Figures 7b, 7d, and 7f have been reproduced as dotted lines in Figure 8. Also shown, as solid lines in Figure 8, are the best fits to the number tracked data for all of the ±19° deviation conditions ( T = 10, 8, and 6), for all of the ±38° deviation conditions, and for all of the ±76° deviation conditions. The actual number of trajectories in the stimulus has only a small effect on the effective number of tracked trajectories over the range of trajectories tested. Likewise, the number of deviating trajectories has only a small effect on the number of tracked trajectories. Most interestingly, it is the magnitude of the angle of deviation that primarily determines how many trajectories are effectively tracked. 

Figure 8

 

The effective numbers of tracked trajectories when the different angles of deviation are blocked. A summary of the effective numbers of tracked trajectories ( E) from Experiment 2 is plotted as a function of the number of deviating trajectories ( D). The figure shows the best-fitting straight lines from Figures 7b, 7d, and 7f, obtained for stimuli with 10, 8, and 6 trajectories, respectively (i.e., for T = 10, 8, and 6). The different colors represent the different angles of deviation. Each solid line represents the best fit to all of the estimates of the effective numbers of tracked trajectories obtained for the corresponding deviation. For ±19° deviation, the slope of the line (green) represents a change of −0.05 trajectories in E per unit change in D. The corresponding slopes for ±38° and ±76° deviations (blue and red lines) represent a change of −0.07 trajectories in E per unit change in D and a change of +0.25 trajectories in E per unit change in D, respectively. The shallowness of these slopes, at least for the two smaller angles of deviation, indicates that the number of tracked trajectories ( E) is largely independent of the number of deviating trajectories ( D).

The effective numbers of tracked trajectories in the above results might reflect actual numbers of trajectories tracked by observers for the different angles of deviation tested. Alternatively, the observers might have used the strategy of judging differences between the average pre- and postdeviation orientations of the array of trajectories to determine the direction of deviation; the increase of the effective number of trajectories tracked with the angle of deviation might simply reflect an improvement in performance on account of the greater availability of cues from the average orientations of the trajectories. The questions that arise from this alternative hypothesis are as follows: Are the average orientations of the pre- and postdeviation trajectories informative with regard to judging the direction of deviation of the targets? If so, under which stimulus conditions are they informative? Did the observers actually use the averaged orientation information in their judgments of deviation direction? If so, how well could they use it? These questions are addressed in the remainder of this section. 

To address the first of the above questions, we performed the following simulation. We generated 200 stimuli similar to those in the experiment, each stimulus having 10 trajectories with one trajectory deviating by ±19°. There were 100 trials with CW deviations and 100 with CCW deviations. For each trial, we determined the average of the predeviation orientations and of the postdeviation orientations, and then we averaged these averages over the 100 trials with CCW deviations. Averaged pre- and postdeviation orientations were similarly obtained for the 100 trials with CW deviations. The averaged orientations for the pre- and postdeviation trajectories for the CCW deviations are shown in green solid lines in Figure 9a; the dashed green lines show the standard deviation of these averaged orientations. The averages for the CW deviations were similar (but not mirror symmetric about the horizontal axis because the standard deviations were large). The averaged pre- and postdeviation orientations and their standard deviations are shown using a similar style in Figure 9a for CCW deviations of 38° (solid and dashed blue lines) and 76° (solid and dashed red lines). Similar average orientations are shown for stimuli with 2 trajectories deviating ( Figure 9b), 5 trajectories deviating ( Figure 9c), and 10 trajectories deviating ( Figure 9d). When there were many trajectories deviating and the angle of deviation was large, the average orientation of the trajectories contained information that could cue the observer about the direction of deviation. 

Figure 9a, 9b, 9c, 9d

 

The average orientation on a trial. Simulations determined the statistical distribution of the mean orientation on a trial before and after the targets deviated. Distributions were determined for trials with CCW and CW deviations separately for a 10-trajectory stimulus, and only the distributions for the trials with CCW deviations are shown here. Solid lines represent the average of the 100 mean orientations from 100 trials with CCW deviations and the dashed lines represent ±1 standard deviation about the average. Green, blue, and red lines represent simulation means (solid lines) and standard deviations (dashed lines) for deviations of −19°, −38°, and −57°, respectively. The distribution of predeviation orientations is shown on the left of each panel and that for the postdeviation orientations is shown on the right. Simulation results for 10 trajectories are shown when (a) 1 trajectory deviated, (b) 2 trajectories deviated, (c) 5 trajectories deviated, and (d) 10 trajectories deviated. For large angles of deviation, the average orientation was markedly different from horizontal only when there were many trajectories deviating.

We next addressed the question of whether cues available in either the left half or the right half of the screen might be adequate to determine the direction of deviation. These cues could include the average orientation information available in either the pre- or postdeviation trajectories as discussed above, motion coherence if many of the trajectories on a trial have similar orientations, and so forth. To address this question, observers SS and ST repeated Experiment 2, once with the left half of the stimulus occluded by a sheet of opaque cardboard and again with the right half of the stimulus occluded. Was the average orientation in either half of the stimulus adequate to determine the direction of deviation? Data were obtained only for the conditions for which effective numbers of tracked trajectories were evaluated in Figure 7. If the information available in either half of the stimulus were inadequate to determine the direction of deviation, then observers' performance should be at chance (50% correct responses). We selected 60% correct response as a cut-off level for determining if usable cues were available in either half of the stimuli; for a particular combination of T, D, and angle of deviation, if either observer had a performance level better than 60% correct in either of the two conditions (left-half occluded or right-half occluded) then, for that combination of parameters, the performance of the observers in the main experiment may have been influenced by additional cues. (The cut-off limit from the binomial theorem should have been 57%. We relaxed this limit because of the repeated comparisons involved and because we were overly conservative in other aspects—the cut-off limit was considered to have been exceeded if any of the four percentages obtained (percent correct for either of the two observers under either of the two occlusion conditions) exceeded the cut-off limit.) Conditions for which the performance was better than the cut-off level, that is, for which extraneous cues may have influenced judgment, are identified in Figures 7b, 7d, and 7f by filled symbols. Most data obtained with targets having a deviation of ±76°, or with more than three trajectories that deviate by ±19° or more, could potentially have been influenced by extraneous cues available. Hence, for some of the experimental conditions, Figure 7 may overestimate the number of trajectories tracked and should be seen as the upper limit to tracking performance in our task. 

The next question of interest is, How effectively can the human observers average the orientations of the trajectories? An additional control experiment was performed to address this question. The stimulus consisted of 6, 8, or 10 trajectories (blocked) as before. Two of the trajectories, the targets, deviated by ±38°. Unlike the experiments described so far, the distractors too deviated, half of them CW, the other half CCW. The magnitude of the angle of deviation of the distractors was 0°, 19°, 38°, 57°, and 76° (blocked). Other details were similar to those in the main experiment and the percent correct responses were determined. Only SS and ST participated in this experiment. It was anticipated that if human observers were basing their judgment on the average orientation of the trajectories on a trial (for an example of orientation averaging in the context of static crowded stimuli, see Parkes, Lund, Angelucci, Solomon, & Morgan, 2001), then the magnitudes of the distractor deviations would have no influence on the observer's performance because the average pre- and postdeviation orientations would remain the same with the CW distractor deviations negating the CCW distractor deviations. Figure 10 plots the percentage of correct responses as a function of the magnitudes of deviations of the distractor trajectories. Open and closed symbols represent data for observers SS and ST, respectively, and the colors blue, green, and red represent the data for 6, 8, and 10 trajectories, respectively. The data have been fitted with exponential curves; observer ST's data for six trajectories were better fitted by a straight line. Perfect averaging would predict that each set of data would fall along a straight, horizontal line. It is apparent from Figure 10 that human observers did not perform optimally when the task required that trajectory orientations be averaged. We expect that human observers will be unable to optimally average trajectory orientation for any of the data represented by open symbols in Figures 7b, 7d, and 7f. A simple averaging of orientations hypothesis cannot explain the effect that the angle of deviation has on the effective number of tracked trajectories (Figures 7 and 8). It remains to be seen whether a more sophisticated averaging hypothesis, possibly involving the variance in the orientations of the trajectories (e.g., along the lines of Sekuler, Sekuler, & Sekuler, 1990), and perhaps involving attention, memory, or both might explain this relationship better. 

Figure 10

 

Results from the control experiment testing the orientation-averaging hypothesis. Half of the distractors deviated CW by a fixed angle and the other half deviated CCW by the same angle. There were two target trajectories among the 6, 8, or 10 trajectories presented, and the angle of deviation of the target trajectories was always ±38°. The percentage of correct responses of observers is plotted as a function of the magnitude of the angle of deviation of the distractor trajectories. The open and filled symbols represent data for observers SS and ST, respectively, and the colored lines represent exponential fits to the data (except ST's six-trajectory data that was fitted with a straight line). The different colors represent data for different numbers of trajectories (T). The percentage of correct responses drops as the magnitude of distractor deviation increases, suggesting that observers' performance was not consistent with the orientation-averaging hypothesis.

Experiment 3: D-of- T trajectories deviating: conditions interleaved

The third experiment was similar to the second but with the different experimental conditions interleaved within each block. As a consequence of our findings from the control experiments in the previous section, we restricted the range of values of D and the range of the angles of deviation to ensure that the extraneous cues in the two halves of the stimulus did not influence the observers judgment of trajectory deviation. At the start of any trial, the observer was unaware of the number of deviating trajectories, or the angle of deviation, and could not preallocate resources to each trajectory based on the difficulty of the task. The difficulty of the task varied from trial to trial because of the random interleaving of the stimulus conditions. Under these circumstances, one anticipates that the effective number of tracked trajectories for the six stimulus conditions interleaved within a block will be fixed at some mean level; that is, unlike in Experiments 1 and 2, the effective number of tracked trajectories will be independent of stimulus conditions, in particular independent of the angle of deviation. 

Figure 11a plots the effective number of tracked trajectories for the three observers for the experimental condition for which the three interleaved angles of deviation were equally likely on any trial within a block. The format used for this figure is the same as that in Figure 7b. We had anticipated that the three straight-line fits to the data for the three magnitudes of deviation angle would be horizontal and would overlap each other. Contrary to expectation, interleaving the angles of deviation had little or no influence on performance. The effective numbers of tracked trajectories when the conditions were interleaved were qualitatively and quantitatively similar to those when the conditions were blocked (compare matched conditions in Figure 7b), and the straight-line fits were not overlapped, as is obvious from Figure 11. On average, the effective number of tracked trajectories for deviations of ±19°, ±38°, and ±57° were 1.28 ± 0.40, 2.27 ± 0.46, and 2.99 ± 0.81, respectively. 

Figure 11a, 11b

 

The effective numbers of tracked trajectories when the different angles of deviation are interleaved. The effective numbers of tracked trajectories ( E) from Experiment 3 is plotted as a function of the number of deviating (target) trajectories ( D), when there are a total of 10 trajectories ( T = 10). The different colors represent the different angles of deviation and the different symbols represent different observers. Panel a shows the data for the condition where on any trial each of the three angles of deviation was equally likely. Panel b shows the data for the condition where deviations of ±38° and ±19° were two times and three times more likely than ±57° deviations. In both cases, effective numbers of tracked trajectories ( E) are similar to those when the different deviations were blocked ( Figures 7 and 8). Average values for the effective number of tracked trajectories were about 1.3, 2.4, and 3.3 for deviations of ±19°, ±38°, and ±57°, respectively. The effective number of tracked trajectories is primarily influenced by the magnitude of the angle of deviation.

Figure 11b plots the effective number of tracked trajectories obtained for two observers when each block contained 60 trials with deviations of ±19°, 40 trials with deviations of ±38°, and 20 trials with deviations of ±57°. As before, we had anticipated that the three straight-line fits to the data for the three magnitudes of deviation angle would be horizontal and would overlap each other. We had further anticipated that the horizontal lines would be shifted downwards compared to when the three magnitudes of deviation angle were equally likely; a greater probability of having a small angle of deviation would cause the observer to attend more carefully to fewer of the trajectories presented. Again, contrary to expectation, the effective numbers of tracked trajectories were qualitatively similar to those obtained in Experiment 2 (compare matched conditions in Figure 7b). On average, the effective numbers of tracked trajectories for deviations of ±19°, ±38°, and ±57° were 1.31 ± 0.41, 2.61 ± 0.24, and 3.57 ± 0.80, respectively. The similarity between the results of the two experimental conditions in this experiment ( Figures 11a and 11b) and the results of the main condition in Experiment 2 ( Figure 7b) argue against the preallocation of resources to the trajectories based on the expected difficulty of the task. Something intrinsic to the magnitude of the angle of deviation determines the effective number of trajectories that can be tracked in our task and this is currently under investigation. 

General discussion

The effective number of tracked trajectories

Pylyshyn and Storm (1988) showed that human observers could track four or five trajectories reliably in what is now a standard MOT paradigm. Tripathy and Barrett (2004) showed that in a threshold paradigm that required human observers to detect the deviations in trajectories, observers were unable to track more than a single trajectory effectively. To understand the difference between the two paradigms, we performed three experiments to estimate the number of trajectories that human observers track when required to discriminate deviations in bilinear trajectories presented simultaneously with several linear trajectories. 

In order to describe human tracking performance quantitatively, we introduced the idea of an LCHO that tracked a fixed number of trajectories perfectly and totally ignored the remaining trajectories. By comparing the performance of the human observer with that of this LCHO, we were able to estimate the effective number of trajectories tracked by the human observer under a variety of conditions. Using the effective number of trajectories to quantify human performance offers a major advantage over using the percentage of correct responses, and this can be illustrated by the following example. Consider two stimuli, each consisting of 10 trajectories, but the first having one deviating trajectory and the second having four deviating trajectories, all of the deviations being ±19°. The percentage of correct responses is a poor indicator of tracking performance. A higher percentage of correct responses for the second stimulus does not necessarily imply that tracking performance was better for that stimulus, it could simply reflect the fact that the probability of detecting a trajectory deviation is higher when there are four trajectories deviating than when there is only one trajectory deviating. The effective number of tracked trajectories takes these different probabilities into account, thus permitting a direct comparison of performance across the different stimulus conditions. 

In the first experiment, we varied (between blocks) the total number of trajectories presented, keeping the number of deviating trajectories fixed at one. When the magnitude of the target's deviation was ±19°, the performance of the observers was comparable to that of an LCHO that perfectly tracked one to two of the trajectories presented and ignored the remaining trajectories; that is, the effective number of trajectories tracked by the observers was one to two. For a target deviation of ±38°, the effective number of trajectories tracked was two to three. When the deviation was ±76°, the effective number of trajectories tracked was three to four. 

In the second experiment, we studied the effective numbers of tracked trajectories more systematically. We estimated the effective number of tracked trajectories when the number of deviating trajectories was varied (between blocks), keeping the total number of trajectories fixed. Estimates were obtained with the total number of trajectories fixed at 10, 8, and 6, and these resulted in the several interesting observations described below. 

Our first interesting finding from Experiment 2 was that when the number of trajectories presented was six or more, the actual number of trajectories presented had only a small influence on the effective number of tracked trajectories. This can be seen in Figure 8, with the best-fitting lines to the (average) effective number of tracked trajectories for T = 10, 8, and 6 for the ±76° deviation data all lying close to one another. This was found to be more the case when the deviation was ±38° and was most evident when the deviation was ±19°. When the number of trajectories in the stimulus was changed from 6 to 10, the corresponding change in the effective number of tracked trajectories was typically much less than one. The fact that the effective number of tracked trajectories did not change proportionately when the number of trajectories in the stimulus was changed from 6 to 10 suggests that, even when only six trajectories were presented, observers were only tracking a subset of the available trajectories. 

Our second interesting finding from Experiment 2 was that the number of deviating trajectories had only a small effect on the effective number of tracked trajectories. A stimulus with one deviating trajectory is very different from one with seven deviating trajectories. Yet, surprisingly, for both of these stimuli, for a deviation of ±19° (and to a lesser extent for a deviation of ±38°) there was very little difference in the effective number of tracked trajectories ( Figure 8). Control experiments suggested that extraneous cues were available to the observers when there were many trajectories deviating (filled symbols in the three panels on the right in Figure 7). However, it is not clear to what extent these cues were used. Therefore, these effective numbers of tracked trajectories should be seen as upper estimates of tracking performance—if these extraneous cues could somehow be removed, the effective number of trajectories might have dropped as the number of deviating trajectories was increased. An important point to note is that the percentage of correct responses increases substantially as the number of deviating trajectories increases. However, as the number of deviating trajectories is increased, the effective number of trajectories, under most of the conditions where adequate data are available (i.e., when the deviation was 19° or 38°), either stays the same or decreases slightly. 

The most interesting finding from Experiment 2 is the substantial effect of the angle of deviation on the effective number of tracked trajectories. The performance averaged across the three observers yielded an effective number of tracked trajectories of slightly more than one for a ±19° deviation, slightly more than two for a ±38° deviation, and close to four for a ±76° deviation ( Figure 8). The angle of deviation was the primary factor that determined the effective number of tracked trajectories. Over a range of values for T and D, the tracking performance of human observers was comparable to a hypothetical observer that perfectly tracked about one (±19° deviation), two (±38° deviation), or four (±76° deviation) trajectories and ignored the remaining trajectories. It is possible that when the values of D were greater than three, the observers' performance might have been compromised by extraneous cues. However, even when these extraneous cues were unavailable, the effective number of tracked trajectories increased systematically with increasing angle of deviation. Could this almost proportional increase in the effective number of trajectories with increase in the angle of deviation have resulted from observers averaging the orientations of the trajectories, both pre- and postdeviation, and basing their judgment on the difference between the two? Such an explanation is unlikely; when observers were encouraged to average the orientations of the trajectories by having half the distractors deviate CW by a fixed angle and the other half deviate CCW by the same angle, performance dropped rapidly when the magnitude of this angle was increased, in spite of the distractor deviations resulting in no change in the average pre- or postdeviation orientation ( Figure 10). 

An obvious interpretation of the results in Experiment 2 is that observers adjust the number of trajectories they track based on the magnitude of the angle of deviation of the target trajectories. If the deviation is large then, at the start of each trial, the available resources can be distributed over a larger number of trajectories; but if the deviation is small, the resources can be distributed among fewer trajectories so that these few trajectories can be processed in greater detail. Because the different experimental conditions were blocked in Experiment 2, such preallocation of resources was potentially possible. In Experiment 3, the different experimental conditions were interleaved and preallocation of resources was no longer possible because the observer did not know the angle of deviation and the number of deviating trajectories at the start of each trial. Under these conditions, one assumes that the observer would assume some average level of difficulty and accordingly distribute the available resources among the appropriate number of trajectories for this level. If this is the case, one anticipates that the effective number of tracked trajectories will be independent of the angle of deviation. Our findings were totally contrary to our expectations; the effective numbers of tracked trajectories were still strongly influenced by the angle of deviation of the targets. Qualitatively, the effective numbers of tracked trajectories when the experimental conditions were interleaved were similar to those when the experimental conditions were blocked. These findings rule out an explanation for the effective numbers of tracked trajectories that is based on the preallocation of resources at the start of each trial to suit the task difficulty. 

The actual explanation for the relationship between the angle of deviation and the effective number of tracked trajectories remains unclear. However, our data permit us to eliminate some potential explanations. One can hypothesize that the deviating trajectories “pop-out” from among the distractor trajectories, and the probability that any target trajectory will “pop-out” from the distractor trajectories is determined by the angle of deviation of the trajectory. Pop-out experiments in visual search typically involve measures of reaction time, and the characteristic signature of “pop-out” is performance that is devoid of set-size effects (e.g., Treisman & Gelade, 1980). In the current experiments, performance was measured using percentage of correct responses and one might anticipate that pop-out would be signaled by an absence of set-size effects on the percentage of correct responses. However, the set-size effects in Figure 6 clearly run counter to these expectations. Performance drops rapidly as the number of distractors is increased, a result which is not consistent with the pop-out hypothesis. Simple crowding of the target trajectory or trajectories or simple masking of the target trajectory or trajectories will also fail to explain tracking performance in the current task; performance is controlled by attentional factors and observers are very sensitive to detecting deviations in trajectories if they are cued to the deviating trajectory at the start of the trial (Tripathy & Barrett, 2004). 

As has been pointed out before (Hulleman, 2005; Tripathy & Barrett, 2004), our task does not require observers to track the moving dots for the entire lengths of the trajectories. It might be adequate to track just the central parts of the trajectories, where the deviation was known to occur. The perceived distortions in the trajectories of the deviating dots suggest that observers only track parts of some of the trajectories (Tripathy & Barrett, 2003, 2006). If only parts of trajectories are tracked, then the effective number of tracked trajectories yielded by our experiments tells us about the limitations on tracking performance when the trajectories were close to the point of deviation. Whether entire trajectories or only parts of trajectories were tracked in our task, our results suggest that, for deviations of the magnitude used in our experiments, human observers behave as if they are allocating adequate computational resources to only a few of the trajectories (between one and four). If it is possible that only parts of trajectories are tracked, then what does “perfect tracking” mean? As used in the current manuscript, perfect tracking does not imply that the internal representation of the trajectory perfectly matches the actual trajectory, or that all of the information regarding the trajectory is stored. What it does imply is that adequate information has been accumulated about the trajectory to make a perfect discrimination of deviation direction, when there is a deviation (of say 19°) in the trajectory. 

Although the results of the study have been discussed from an attentional perspective, the findings have more general implications. Equations 1, 2, 3 and 4 do not have any representation for the angle of deviation. A representation of the angle of deviation is not necessary because, for the magnitude of deviations used, we presume that the hypothesized deviation discrimination is perfect if resources (attentional, computational, etc.) are directed toward this trajectory. In other words, the total information utilized by the human observer is comparable to the information available in (approximately) one, two, or four trajectories when the deviation is ±19°, ±38°, or ±76°, respectively; that is, information from at least one, two, or four trajectories must be utilized in order to match human performance. If one prefers an attentional explanation for the current findings, then at least one, two, or four trajectories must have been attended in order to explain the performance obtained for ±19°, ±38°, or ±76° deviations. Of course this does not rule out that attention may have been directed to more trajectories than this minimum number. If one prefers an alternative explanation to the current findings, such as an averaging of orientations explanation, then the current findings are still informative—the orientations of at least one, two, or four trajectories must be averaged (perfectly) in order to explain the results for ±19°, ±38°, or ±76° deviations, respectively. However, the poor averaging performance indicated by Figure 10 suggests that the actual number of trajectories averaged will have to be greater than these minimum numbers (if trajectory orientations are indeed averaged). 

Visual indexes

Pylyshyn has proposed that a set of “visual indexes or FINST (Fingers of Instantiation)” is used for tracking objects in motion (e.g., Pylyshyn, 1989; Pylyshyn, 2000, 2001; Pylyshyn et al., 1994; Pylyshyn & Storm, 1988). Each of these indexes is attached to a moving object and moves with the object. If a set of visual indexes is used to keep track of the moving dots in our experiments, then our results suggest that the number of visual indexes available can vary between one and four when the task is to detect deviations in trajectories and this number is influenced by the difficulty of the task, that is, in our case by the angle of deviation of the trajectories. In addition, the number of visual indexes is only slightly influenced by the total number of trajectories actually presented and the number of deviating trajectories. 

Target discriminability and set-size effects

Intuition suggests that set-size effects in attentional tasks could be eliminated by making the target(s) highly discriminable from the distractors. In our task, our target trajectories were lines with a single deviation and the distractor trajectories were undeviating lines. A deviation of ±19°, in our opinion, is highly discriminable. However, to ensure that the target with a ±19° deviation is highly discriminable, we repeated Experiment 1 with a single trajectory (target alone) with a ±19° deviation. Ten blocks were run for each observer, yielding a total of 1000 trials. All three observers correctly identified the direction of deviation on 1000 of the 1000 trials. There is little doubt that a +19° deviation in a trajectory is highly discriminable compared to a −19° deviation (and also a 0° deviation). However, set-size effects are clearly evident for the ±19° deviation data shown in Figure 6, with the average performance dropping from 100% for a single trajectory to less than 60% when 10 trajectories are present. The intuitive expectation that set-size effects are eliminated when the target is highly discriminable does not appear to hold for our task of detecting deviations in trajectories (or for MOT tasks in general). 

Traces in memory

Our effective number of tracked trajectories could be an indication of the number of trajectories that can be tracked at a time; that is, the number of trajectories for which sufficient information is available in some internal representation. In other words, for E trajectories the internal representation could contain adequate information for discriminating the direction of deviation and for the remaining ( T − E) trajectories the information available might be inadequate for deviation discrimination. But an alternative explanation based on the transience of the traces of the trajectories is also plausible. In Narasimhan, Tripathy, and Barrett (2005, 2006b), we measured deviation thresholds along the lines of Tripathy and Barrett (2004), but we cued the target trajectory during the second half of the display to examine if the earlier parts of the cued (target) trajectory could be remembered. Thresholds were only slightly improved by cueing, suggesting poor recall of the early parts of the trajectories (for a similar experiment with static letters instead of trajectories, see Sperling, 1960). In another experiment, we introduced a delay between the first and second halves of the display on each trial (see Sperling, 1960, who used a similar technique to demonstrate the temporal characteristics of visual sensory memory or iconic memory for tachistoscopic stimuli with static letters). When there were three trajectories in the stimulus, a delay of 300–400 ms halfway through the trial resulted in thresholds being elevated by a factor of four in some observers. This deterioration of performance, similar to that seen in Sperling's (1960) experiments, suggests the involvement of visual sensory memory in tracking; trajectory traces in memory may be utilized by the visual system in these deviation detection tasks. The decision process could involve the sequential accessing of these rapidly decaying traces to determine the direction of deviation of the target(s). The effective number of tracked trajectories may be a reflection of the number of trajectory traces that can be scanned before they decay to the point of not being useful to the decision process anymore. Although both attentional and memory-based explanations are plausible, we currently favor the explanation that is based on the scanning of traces in memory; such an explanation would be consistent with the current findings and the findings of our other recent studies that suggest the involvement of memory in deviation detection with similar stimuli (Narasimhan, 2006; Narasimhan et al., 2005, 2006b). Interestingly, amblyopes show little or no deficit in tracking deviations in trajectories, whether the deviations are close to threshold (Levi & Tripathy, 2006a, 2006b) or are substantially suprathreshold, as in the current study (Tripathy & Levi, 2006a, 2006b, 2006c). Amblyopes have been shown to undercount features in a stimulus, suggesting a deficit in attentional processing when viewing with their amblyopic eye (Sharma, Levi, & Klein, 2000). The absence of an amblyopic deficit in the current task suggests that the limits to performance in the current task are probably not attentional. 

Summary

Over the range of parameters tested, we found that the effective number of tracked trajectories was

1.  
   only slightly affected by the number of trajectories presented;
2.  
   only slightly affected by the number of deviating trajectories; and
3.  
   substantially affected by the angle of deviation of the trajectories.

Over this range of parameters, the human observers' performance was, for all practical purposes, as if they were tracking a subset of the trajectories and ignoring the remaining trajectories. The effective number of tracked trajectories varied between one and four, depending on the magnitude of the angle of deviation of the target trajectories. Simple hypotheses based on “averaging of orientation,” “preallocation of resources,” pop-out, crowding, or masking are unlikely to provide a satisfactory explanation for this relationship between the effective number of trajectories tracked and the angle of deviation. Explanations based on visual resolution are also unlikely to explain our findings because in the current task, whether using threshold stimuli or suprathreshold stimuli, amblyopic eyes perform as good as the eyes of normal observers (Levi & Tripathy, 2006a, 2006b; Tripathy & Levi, 2006a, 2006b, 2006c). However, at the moment, we cannot rule out complex hypotheses involving one of more of the above explanations, perhaps also involving memory, attention, or both. 

In our experiments, the stimuli were dynamic, consisted of multiple moving dots, and the observers were required to report the deviation of a subset of these dots. When comparing our results with previous studies from the literature, it is not apparent to what extent earlier explanations for human performance with static stimuli (e.g., Sperling, 1960; Parkes et al., 2001), with moving stimuli involving single objects (e.g., Sekuler et al., 1990), or with multiple moving dots where the observer is required to report the global percept and not details of individual dots (e.g., Watamaniuk, Sekuler, & Williams, 1989) can be generalized to our current experiments. Much experimental work still needs to be done before we have a good understanding of human performance in the current task and can tie these explanations to those proposed for performance with stimuli very different from ours. It is likely that explaining human performance in the current task will require a different approach from those used in the earlier studies. 

Acknowledgments

Experiment 1 was presented at Vision Sciences Society meeting at Sarasota (Tripathy, 2003). Experiment 2 was presented at the European Conference on Visual Perception (ECVP) at La Coruna (Tripathy, Narasimhan, & Barrett, 2005). Experiment 3 was presented at ECVP at St. Petersburg (Narasimhan, Tripathy, & Barrett, 2006a). The experiments presented here formed part of Sathyasri Narasimhan's PhD dissertation (Narasimhan, 2006). We thank Oliver Braddick, Dennis Levi, Haluk Ogmen, and Ozgur Yilmaz for their thoughtful comments on the manuscript. Oliver Braddick and Andrew Watson independently proposed Experiment 3 and two anonymous referees proposed some of the control experiments described under Experiment 2; these suggestions have led to a substantial improvement of the paper and are much appreciated. Author SN's current address: College of Optometry, University of Houston, 505 J. Davis Armistead Building, Houston, TX 77204-2020, USA. 

Commercial relationships: none. 

Corresponding author: Srimant P. Tripathy. 

Email: [email protected]. 

Address: Department of Optometry, University of Bradford, Richmond Road, Bradford BD7 1DP, UK.