Taking a(c)count of eye movements: Multiple mechanisms underlie fixations during enumeration

Abstract

We habitually move our eyes when we enumerate sets of objects. It remains unclear whether saccades are directed for numerosity processing as distinct from object-oriented visual processing (e.g., object saliency, scanning heuristics). Here we investigated the extent to which enumeration eye movements are contingent upon the location of objects in an array, and whether fixation patterns vary with enumeration demands. Twenty adults enumerated random dot arrays twice: first to report the set cardinality and second to judge the perceived number of subsets. We manipulated the spatial location of dots by presenting arrays at 0°, 90°, 180°, and 270° orientations. Participants required a similar time to enumerate the set or the perceived number of subsets in the same array. Fixation patterns were systematically shifted in the direction of array rotation, and distributed across similar locations when the same array was shown on multiple occasions. We modeled fixation patterns and dot saliency using a simple filtering model and show participants judged groups of dots in close proximity (2°–2.5° visual angle) as distinct subsets. Modeling results are consistent with the suggestion that enumeration involves visual grouping mechanisms based on object saliency, and specific enumeration demands affect spatial distribution of fixations. Our findings highlight the importance of set computation, rather than object processing per se, for models of numerosity processing.

Introduction

There is strong evidence for a specialized fronto-parietal network for processing numerosity. Posterior parietal cortex of both humans and nonhuman primates appears to encode small numerosities (≤ four; Harvey, Klein, Petridou & Dumoulin, 2013; Nieder, 2016), while the prefrontal cortex is implicated in symbolic number encoding in primates (Diester & Nieder, 2007) and mental calculation in humans (see Arsalidou & Talyor, 2011). Neural circuits in inferior parietal regions that subserve numerosity processing overlap with those involved in eye movements and calculation (Simon, Mangin, Cohen, Bihan, & Dehaene, 2002). Although enumeration of larger set sizes (numerosity greater than four) in humans invariably involves eye movements, it is not known whether enumeration eye movements are directed by the fronto-parietal numerosity network or by obligatory visual processing not specific to enumeration. 

Complex fixation patterns are often observed during enumeration. A consistent finding is the number of fixations correlates with set size (Chi & Klahr, 1975; Schleifer & Landerl, 2011). Since response times similarly increase with set size, it is possible that the association between the number of fixations and numerosity is simply the result of longer viewing time. Other research suggests that eye movements are unlikely to reflect counting of individual elements within a set though, since fixations tend to cluster near the centroid of multiple elements (Li, Logan, & Zbrodoff, 2010). Moreover, attentional demands of enumeration are greater than sequentially fixating each object in an array (Wilder, Kowler, Schnitzer, Gersch, & Dosher, 2009). Given the equivocal conclusions of previous research, it remains unclear what specifically can be inferred about the nature of numerosity processing from patterns of eye movements during enumeration. 

Enumeration fixation patterns have been attributed to visual search and object-oriented processing (Sophian & Crosby, 2008; Watson, Maylor, & Bruce, 2007) that prioritize individuation of dots. However, enumeration differs from visual search in two critical ways: (a) visual search has clearly defined exploration/termination rules (e.g., target absence/presence), and (b) only previously visited coordinates need to be retained during search, whereas quantity information also has to be retained during enumeration. Visual search models of eye movements do not align with the computational demands of enumeration since they focus on individual elements as “objects,” whereas numerosity is a property of sets. Measuring fixation patterns in tasks that emphasize set-based enumeration would arguably provide more useful information about the association between eye movements and numerosity processing in enumeration. 

Spatial grouping mechanisms that do not involve individuation have been proposed to account for response times and accuracy during enumeration. Models of these mechanisms are based on the claim that early visual processing is sensitive to the detection of object contours; in particular, that objects are perceived as belonging to small groups on the basis of local proximity cues (Compton & Logan, 1993, 1999; Van Oeffelen & Vos, 1982, 1984). These models of spatial grouping mechanisms are able to explain certain behavioral phenomena (e.g., faster to enumerate grouped arrays), but little research has investigated whether these models can account for the spatial distribution of enumeration eye movements. 

Recent research on approximate numerosity comparison tasks suggests numerosity may be encoded by a dedicated neural mechanism or mechanisms (Anobile, Cicchini, & Burr, 2014; Anobile, Turi, Cicchini, & Burr, 2015; Burr & Ross, 2008; Cicchini, Anobile, & Burr, 2016; Ross & Burr, 2010) or derived from salient visual-spatial features (e.g., texture, density, and area) (Dakin, Tibber, Greenwood, Kingdom, & Morgan, 2011; Durgin, 2008; Morgan, Raphael, Tibber, & Dakin, 2014; Raphael & Morgan, 2015). It is unclear from these studies whether the same visual-spatial mechanisms underlie the spatial distribution of fixations during enumeration of precise numerosities. Whereas eye movements provide high-fidelity spatiotemporal information about perceived visual saliency of sets of objects (Schütz, Braun, & Gegenfurtner, 2011), little is known about the specificity of patterns of fixation during enumeration. 

Here we tested two predictions about enumeration fixations patterns. First, if eye movements are contingent upon the interdot statistics of the arrays to be enumerated (e.g., density, proximity), oblique rotations applied to arrays (i.e., affine transformations that do not alter the configuration of dots) should result in corresponding rotations of fixation locations. Furthermore, highly similar fixation patterns would also be expected if the same arrays were presented multiple times. Alternatively, if eye movements are not related to the specific dot configuration, we predict that individuals may fixate absolute image coordinates (e.g., top-left, bottom-right) irrespective of interdot statistics. 

To test the first prediction, participants enumerated random dots arrays. Arrays were shown in one of four orthogonal orientations on separate trials (0°, 90°, 180°, and 270°), with some arrays shown on multiple occasions to assess consistency in fixation patterns for the same dot configuration. We assessed invariance in eye movement patterns by comparing the overlap in the spatial density of fixations between trials. An information-theoretic metric quantified the similarity of spatial fixation distributions in two dimensions (i.e., horizontal and vertical spatial axes). 

Insofar as the first prediction is correct, shifts in fixations patterns following the rotation of dot locations would be consistent with the claim that participants enumerate arrays by identifying salient groups of dots and integrating the cardinality of each group (Camos, 2003; Starkey & McCandliss, 2014). Specifically, our second prediction is that fixation patterns should be similar when participants view the same arrays from the first task but instead enumerate the number of groups of dots they perceive. A simple pooling model of individual elements was used to derive visual saliency maps (i.e., image intensity) and identify distinct groups of dots in the arrays. Various filter sizes were examined to model differences in object saliency across increasingly larger regions of the arrays (Schütz et al., 2011). The number of distinct salient regions identified for each array was then compared with participants' subset judgments and fixation patterns. 

Materials and methods

Participants

Twenty adults participated (15 female, five male; M = 20.15 years, SD = 3.34). All had normal or corrected-to-normal vision. The study adheres to the Declaration of Helsinki and was approved by the author's University's Human Research Ethics Committee. 

Tasks and procedure

In the Dot Enumeration task participants enumerated the number of dots in each array. Participants were shown the same arrays in the Subset Enumeration task and reported the number of sets of dots they perceived. Both tasks began with a central fixation cross, shown for 500 ms, to signal the beginning of a trial. Dot arrays remained visible until participants pressed a response button (RTbox; Li, Liang, Kleiner, & Lu, 2010). The screen cleared and a small black square appeared centrally for 2000 ms to signal the need to give a verbal response. No feedback was provided and the next trial was self-initiated. The same pseudorandom trial order was used for every participant. The dot enumeration and subset enumeration tasks were administered in separate 45-min sessions on different days. 

Stimulus design

Arrays were constructed for dot set sizes n = 1–12 by dividing a circular region at the center of the display (diameter subtending 15° visual angle) into n equally-sized segments. The distance of each dot from the center of the display was sampled from an inverse cumulative distribution (i.e., square-root of the radius) to ensure equally probable density across the entire circular region. The angular position of each dot was Gaussian distributed from the center of each segment to avoid regular radial patterns. This sampling procedure was repeated until all pairwise distances between dots were greater than a visual degree to prevent occlusion and reduce the likelihood of crowding. Two unique dot arrays were constructed for each set size. 

The experimental design involved two array manipulations. First, each dot array was shown at four orthogonal orientations (0°, 90°, 180°, and 270°), resulting in eight trials in total for each set size. Two dot arrays (one each of set sizes eight and nine) were also presented for an additional four trials each to assess the consistency of fixation patterns. All participants were shown the same dot arrays to assess similarity in fixation patterns. All arrays were generated and displayed using Psychtoolbox and Eyelink Toolbox extensions in MATLAB (Cornelissen, Peters, & Palmer, 2002; Kleiner et al., 2007). Dots were 12 pixels or approximately 0.5° in diameter, displayed in black [RGB (0, 0, 0)], on a gray background [RGB (128, 128, 128)], with a screen luminance of 14.7 cd/m² (xy = 0.299, 0.338). 

Apparatus

Arrays were presented on a 22-inch LaCie Electron CRT monitor with a 1024 × 768 pixel resolution and refresh rate of 120 Hz. Participants sat approximately 57 cm away from the monitor (1 cm subtended 1° visual angle), with forearms resting on a flat surface and viewed displays from a chinrest to reduce head movements. An EyeLink II head-mounted tracker (SR Research, Canada) sampled fixation location via pupil position of the right eye at 250 Hz. A 9-point (3 × 3 grid) calibration/validation routine was conducted prior to testing; participants fixated a series of target locations in a pseudorandom order. This routine was repeated until fixation-position errors were within the spatial resolution of the eye tracker with average deviation < 0.5°, maximum deviation < 1°. Trials were self-paced to ensure eye tracking fidelity and reduce the likelihood of blink artefacts. A drift correction preceded each trial to realign fixation position coordinates. Eye movement parameters were extracted using Eyelink saccade detection algorithms with default thresholds. Saccade and fixation events shorter than 100 ms occurring immediately pre- and postblink events were discarded. 

Fixation spatial density analysis

The spatial distributions of fixations were quantified by fixation density maps (for example, see Le Meur & Baccino, 2013). For N_i observers with M_k fixations at x spatial coordinates, a probability density for the distribution of fixations can be defined for an individual (Equation 1) and the entire sample (Equation 2):  

 

 

 

 

The δ value represents Kronecker delta in Equation 1 (i.e., δ(x) = 1 for each fixated coordinate, otherwise δ(x) = 0), and was scaled by fixation duration in all analyses to quantify processing time. A saliency map S is computed by pointwise multiplication of the fixation probability density map f with an isotropic bidimensional Gaussian kernel G (see Equation 3):  

 

 

The free parameter σ represents the standard deviation of the Gaussian kernel, which was set to approximately one degree visual angle as an estimate of central fovea acuity. Saliency map intensity values were normalized within conditions for comparison. Image saliency maps for each dot array were calculated using the same logic. Each array was first converted into a 768 × 768 grayscale image before being multiplied pointwise by a Gaussian kernel with σ = 1° (see Equation 3) to produce a smoothed saliency map. 

To measure the deviation between any two fixation saliency maps, P and Q, the Jensen-Shannon divergence measure was calculated (D_JS, derived from the Kullback-Leibler divergence D_KL; see Equation 4):  

 

where D_KL(P‖Q) = ∑_iP(i)log (P(i)/Q(i)) and M = 1/2 (P + Q).  

Similarity between fixation saliency maps was defined as the inverse of the Jensen-Shannon distance (1 – square-root of D_JS). The JS-distance is preferred over the KL-divergence because of it metric properties that help with interpreting similarity: Comparisons between fixation maps are symmetric and bounded between [0–1], with values closer to zero representing lower similarity in the dispersion of fixations (and vice versa). 

Results

Relationship between time taken to enumerate and report the perceived number of sets

The response times for all trials are plotted in the top panel of Figure 1 (A: linear scale; B: log-transformed). Each point in the bivariate scatter plots represent a paired trial from the Dot and Subset Enumeration tasks where the same array was presented. A significant linear trend is evident from the log-transformed RTs, reflecting an estimated 53.56% shared variance (adjusted-R²); in other words, over half of the variance in RTs was common despite enumeration of different aspects of the same array. Importantly, overall accuracy for the Dot Enumeration task was high (mean = 0.943, range = 0.702 – 1.00) and participants gave responses to most trials in the Subset Enumeration task (mean = 0.989, range = 0.952 – 1.00). Trials were discarded from analyses if a response was made in less than 200 ms or more than 10,000 ms from trial onset (2.64% and 2.93% of Dot Enumeration and Subset Enumeration trials, respectively). 

Figure 1

 

Bivariate scatter plots with estimates of accounted variance. Each point represents a trial from the Dot Enumeration (x axis) and Subset Enumeration (y axis) tasks where the same arrays where shown. (A), (B) Linear-linear and log-log scaled response times, respectively. (C) Number of saccades made on trials of identical dot arrays across tasks (scatter point size is scaled by frequency). (D) Mean fixation durations for trials of identical dot arrays across tasks. Line of equivalence (——) and least squares regression fit (- - - -).

Consistent with previous research, there was evidence of a strong, significant association between set size and number of fixations for Dot Enumeration (Kendall rank correlation τ = 0.574, p < 0.001), while the number of fixations was also moderated correlated with the perceived number of subsets the Subset Enumeration task (Kendall rank correlation τ = 0.436, p < 0.001). Outliers were removed before calculating these correlations (i.e., trials with a fixation count three times greater than the median absolute deviation value; 62.92% and 71.04% of Dot Enumeration and Subset Enumeration trials were analyzed, respectively). 

The number of saccades made per trial and mean fixation duration were examined to evaluate the possibility that obligatory visual processing of the same arrays could account for similarities in RT (see Figure 1C and D, respectively). When presented with identical dot arrays, participants made more saccades to enumerate dot arrays, M (SD) = 5.86 (5.04), compared to judging the number of sets perceived, M (SD) = 4.67 (4.41), paired-sample t test (df = 1816) = 11.779, p < 0.001. There was a moderate correlation between the number of saccades made on trials where the same dot arrays were shown across the two tasks (Kendall rank correlation τ = 0.531, p < 0.001). Mean fixation durations per trial were longer on average for Subset Enumeration, M (SD) = 282.07 (116.24) ms, than Dot Enumeration, M (SD) = 272.87 (91.41) ms, paired-sample t test (df = 1580) = −2.751, p = 0.006. However, there was a low correlation in mean fixation durations across tasks (Kendall rank correlation τ = 0.155, p < 0.001, adjusted-R² = 0.057). Fixations with duration less than 100 ms, and more than 1,100 ms were excluded from this analysis (7.35% and 7.72% of fixations for Dot Enumeration and Subset Enumeration tasks, respectively). 

Systematic tendency for initial saccades to be oriented toward the upper visual field

The first saccades preceding trial onset were analyzed to quantify the tendency for participants to initially orient towards particular image locations. Saccades were often directed towards the upper visual field in both tasks, with an additional leftward trend in the Dot Enumeration task (see Figure 2). Specifically, angular dispersion of first saccades towards any image quadrant significantly deviated from chance for both Dot Enumeration, χ² (1) = 19.6, p < 0.001, and Subset Enumeration, χ² (1) =13.2, p < 0.001. 

Figure 2

 

(A), (B) Mean angular dispersion of first saccade landing positions towards image quadrants for each trial of the Dot Enumeration and Subset Enumeration tasks, respectively. Each dot represents an individual participant, with dot area scaled to reflect the average spatial deviation from the mean saccade landing position. Dot color is proportional to dot size (larger areas are lighter and vice versa), so darker regions reflect partial overlap across participants.

A bootstrap sample of saccades was simulated to derive a baseline for comparison (and to calculate the expected cell counts) from 20 permutations with 96 iterations under four different plausible conditions: saccade towards (a) any random location within the display, (b) the location of the closest dot from central fixation, (c) the location of any random dot in each array, and (d) the most salient location within each image. The initial fixation position was also randomly sampled from the central fixation location with a radius of one visual degree to simulate within-trial variability of tracking. In all cases, the likelihood of the first saccade occurring in any quadrant was equally probable given the dot array density. 

These preliminary analyses suggest a large amount of variance in enumeration response times can be attributed to necessary visual-spatial processing demands, independent of specific task demands. Moreover, the relationship in response times for both tasks could not be explained by differences in temporal characteristics of fixations. We next analyzed the spatial distribution of fixations to verify these conclusions and evaluate if systematic manipulations to dot locations affected fixation patterns. 

Orthogonal rotation and repetition of arrays influenced fixation patterns

Fixation saliency maps for the Dot Enumeration and Subset Enumeration tasks are shown in Figure 3. Of particular interest is the similarity in fixation patterns for identical arrays presented at four orientations and on four occasions (one array each of sets eight and nine). The degree of overlap between repeated and rotated trials was compared to determine (a) the consistency of fixation patterns to identical arrays over time, and (b) whether common image locations were fixated irrespective of the spatial orientation of dot arrays and task demands. 

Figure 3

 

(A–H) Fixation density maps overlaid from the four rotated and repeated trials of set size eight (top row) and nine (bottom row) for both Dot Enumeration and Subset Enumeration tasks. Each image represents the fixation density maps pooled across all 20 participants at 50th percentile intensity threshold. Darker regions reflect greater overlap in fixation density across participants. (I) Pairwise similarity comparisons between all four rotated and repeated trials (sets eight and nine), in reference to baseline similarity (dashed line, calculated as the average similarity in fixation density between two randomly selected trials sampled 1000 times). Data points represent pairwise similarity scores, lines represent means. (J) Circles represent mean ± SE pairwise similarity between fixation density and saliency maps (σ = 1°) as a function of set size for Dot Enumeration (closed circles) and Subset Enumeration (open circles). Diamonds represent pairwise comparison of fixation similarity for the same trial across tasks. Note: (E–H) are rotated to a single orientation to illustrate the degree to which participants fixate absolute image coordinates or relative array locations.

The fixation density maps are plotted at the 50 percentile threshold in Figure 3 to highlight regions most commonly fixated for rotated and repeated trials of the same dot array; the darkest patches correspond to areas with the highest overlap across all participants (note, rotated trials are all plotted in a single orientation so that fixation densities can be compared with repeated trials). Two features are evident. First, the greatest overlap in fixation density between participants was at the center of visual arrays, which is unsurprising given trials began with a central fixation cue. Nevertheless, an overlap in fixation density is less obvious in the rotated trials, which suggests participants' fixations shifted in alignment with the dot array configuration. Second, fixations were spread over a larger area during subset enumeration compared to dot enumeration. This is most apparent for the rotated trials (Figure 3, F through H) and indicates the centroid of multiple dots, rather than precise dot locations, were fixated as individuals judged the perceived number of sets. 

The Jenson-Shannon distance metric was used to measure pairwise similarity in fixation density between trials (see Equation 4). A baseline similarity value was computed first to reflect chance level of fixation overlap given any particular set of dots on screen. The baseline was determined by calculating the average similarity in fixation density between two randomly selected trials over 1000 samples with replacement. A baseline similarity value of 0.421 was identified using this procedure. 

Indeed, repeated trials of both set sizes and for both tasks showed a high degree of overlap (> 0.8; see Figure 3I). In contrast, the spatial fixation distribution on rotated trials was less similar on average and showed greater variability, compared to fixations on repeated trials. To determine whether this pattern of findings held for all other set sizes, three further similarity comparisons were conducted. First, each paired trial of the same dot array across both Dot Enumeration and Subset Enumeration tasks were compared to assess the likelihood that participants scanned arrays in a similar manner irrespective of task demands. Average between-task similarity scores are plotted as a function of set size in Figure 3J (diamonds). Similarity scores were constant across set size, mean slope (SE) = 0.006 (0.002), t(94) = 3.061, p = 0.003, showing a high degree of consistency in scan patterns across participants, irrespective of task. 

Fixation density comparisons were also conducted separately for both tasks to determine if fixations overlapped with dot locations. The visual salience of each dot configuration was estimated based on image intensity using the same method as calculating fixation density (see Equations 1 through 3). Image saliency maps were computed with a spatial filter width approximately the size of central vision (σ = 1°; Strasburger, Rentschler, & Jüttner, 2011) and then compared to fixation density maps of both tasks as a function of set size. 

Overall, fixations were closer to dot locations during dot enumeration than subset enumeration. As can be seen in Figure 3J, average similarity scores increased linearly across sets for Dot Enumeration, open circles: mean slope (SE) = 0.038 (0.002), t(86) = 21.462, p < 0.001, and Subset Enumeration, closed circles: mean slope (SE) = 0.024 (0.002), t(86) = 13.623, p < 0.001. The exception was for set size one, which showed a relatively high overlap between fixations and dot locations indicating that participants tended to look directly at the single dot in those arrays, so was not included in calculating slope coefficients. 

Individual differences in fixation density and influence of image saliency

One explanation for the fixation patterns shown in Figure 3 is that the most salient regions of arrays are more likely to be fixated. It is also possible individuals do not fixate the same locations on every trial and pooled data might obscure individual differences. To examine these possibilities, fixation densities from each trial were estimated separately for each individual in both the Dot Enumeration and Subset Enumeration tasks (see Figure 4, middle and bottom rows respectively). Given inherent differences in density for larger sets, this analysis was conducted separately for small (one–four), medium (five–eight) and large (nine–12) dot arrays (see Figure 4, top row). Image saliency maps are shown at 50th percentile intensity threshold in Figure 4. 

Figure 4

 

(Top row) Composite image saliency maps derived from dot arrays of small, medium and large set sizes presented at 50% threshold. Fixation density maps for the Dot Enumeration (middle row) and Subset Enumeration (bottom row) tasks overlaid for all individuals at 50% threshold. Darker regions reflect image locations with the greatest overlap in fixation density between individuals.

It should be noted that average image intensity differed across small, medium, and large sets (Figure 4A). The 90° rotation of arrays resulted in systematic patterns of saliency given the likelihood of dots was isomorphic in every quadrant (i.e., the relative configuration of dots was not altered, just the absolute spatial position). For small sets, fixations tended to remain close to the central fixation cue even though they were dispersed horizontally in the upper hemifield. There is little apparent difference between tasks for small sets. For medium sets, fixations tended to be distributed across arrays, with a high overlap in two regions to the left and right of the center: This was particularly evident in the Dot Enumeration task. For large sets, a clear pattern is evident for both tasks. For Dot Enumeration, participants tend to avoid the center after the initial fixation cue and fixate three distinct spatial locations (see Figure 4, middle panel). For Subset Enumeration the pattern is less clear, with fixations clustered towards the center of arrays. To confirm these observations, image filtering techniques were used to model visual salience across difference spatial scales. 

Subset enumeration as a function of object saliency

Image filtering analyses were conducted to model variability in the number of sets participants reported as a function of salience (i.e., relative visual distinctiveness of dots). Specifically, a series of spatial filters covering increasingly larger areas of arrays were compared to approximate the size of space individuals perceived as partitioning arrays into salient regions and, ipso facto, sets. These saliency maps reflect the estimated likelihood of dots, or sets of dots, at all locations within an array (see Figure 5). 

Figure 5

 

(A) Grayscale image of trial with set size eight. (B) Image saliency maps of the same dot array multiplied pointwise by a Gaussian filter with widths ranging from σ = 1° − 3.5°, in 0.5° steps. (C) Number of sets reported from all participants for trials of the example dot array shown in (A). (D) Open circles represent the median number of salient regions identified at each filter size as a function of set size. The solid line reflects the mean number of salient regions identified from 1000 randomly sampled arrays for each set size, generated using the same constraints as the arrays in the current study as a baseline for comparison. The gray region reflects ± 2 SD confidence bounds. (E) Overall likelihood of the number of perceived sets for all trials, pooled across individuals. (F) Proportion of the number of salient regions identified for all trials, at various filter sizes.

The spatial kernel width (σ) specifies the region over which saliency is integrated; for instance, a width of 1°–2° visual angle is consistent with the size of foveal vision, while ≤ 8° visual angle typically demarcates the central visual field from peripheral vision (Strasburger et al., 2011). Filter widths of σ = 1°–5° (steps of 0.5°) were used that correspond to the spatial resolution of the eye tracker (lower limit) and the central third of the display area (upper limit). The assumption underlying this modeling is that saccades are directed towards salient regions across central and peripheral vision (Schütz, Trommershäuser, & Gegenfurtner, 2012). 

We examined the relationship between behavioral responses and the number of salient regions identified at various filter widths to determine the size of the arrays perceived as encompassing a set. The number of salient regions identified in each array was calculated as the sum of unique local maxima in the image saliency maps. Maxima were measured by a one-pixel sliding window applied in all directions, centered on every pixel location. An example trial is shown in Figure 5A, and the associated saliency maps for different filter widths can be seen in Figure 5B. A small number of salient regions emerged as filter width increased; however, filters larger than 3.5° were insensitive to interdot groupings and converged to a single salient point, which is not reflective of the behavioral responses. For instance, Figure 5C shows on most trials participants perceived the array in Figure 5A as containing two sets. 

The median number of distinct salient regions identified for each set size at every filter size is shown in Figure 5D (open circles). To provide a baseline for comparison, 1,000 arrays were generated for each set size using the same principles used to create the arrays tested in the study, and were then analyzed using the same image filtering. The solid line in Figure 5D reflects the mean number of sets identified at each filter size for the generated arrays, and the gray region represent 95% confidence interval. All arrays used in the current study (open circles) fall within the gray confidence bounds, which suggest the filtering results generalize to similar dot configurations that cover the same spatial extent. 

We sampled from each filter salience distribution to determine the most likely filter size that could have generated the behavioral responses. Variability in the perceived number of sets for every trial pooled across participants is plotted in Figure 5E. The mean perceived number of sets was 2.321 (SD = 0.985, range = 1–6). Two-sample Kolmogorov-Smirnov goodness-of-fit hypothesis tests were conducted to compare the empirical response distribution and each predicted filter distribution (see Figure 5F). Since the empirical distribution of perceived number of subsets was positively skewed, we computed one-sided KS-test statistics to identify the filter size at which the predicted filter distribution was larger than the empirical distribution. A filter width of 2.5° was the smallest distribution of scores not significantly larger than the empirical response distribution (1°–2°: K-S = 0, all ps > 0.05) and (2.5°: K-S = 0.123; 3°: K-S = 0.330; 3.5°: K-S = 0.456; 4°: K-S = 0.623; 4.5°: K-S = 0.707; 5°: K-S = 0.790, all ps < 0.001). These results show dots located within 2.5° (≤ 2.5 cm) from each other were perceived as belonging to the same set. 

Discussion

The findings identify two important links between enumeration and eye movements. First, fixations are conditional upon the spatial location of to-be-enumerated dots, and second, distinct fixation patterns are evident for different numerosity judgments of the same array. Overall, five findings should be noted: (a) Rotating the spatial coordinates of dots, while maintaining their configuration, leads to systematic shifts in the most commonly fixated locations; (b) there was a high similarity between fixation distributions when the same array was presented on multiple occasions; (c) initial saccades were oriented towards the upper visual hemifield despite an equal likelihood for dots to occur in any quadrant across all arrays; (d) there was a linear relationship between the time required to enumerate the cardinality or the number of subsets in the same array; and (e) the number of salient groups of dots identified with image filtering analyses was associated with the number of subsets reported by participants. These findings provide good evidence to suggest enumeration eye movements are influenced by spatial grouping mechanisms related to the saliency of sets of objects. 

Fixations are contingent on dot locations rather than spatial coordinates

Participants tended to fixate arrays relative to the location of dots rather than spatial coordinates (i.e., top-left, bottom-right). This finding was demonstrated by a consistent shift in fixation patterns following 90° orthogonal rotation of the dot arrays. Moreover, there was a high degree of overlap in the spatial distribution of fixations on repeated trials of the same array, suggesting salient visual features of arrays (i.e., spatially distinct groups of dots) may influence how individuals scan arrays to accumulate numerical information. 

Eye movement patterns differed as a function of set size. Fixations tended to occur near the centroid of dots for small set sizes, but tended to cluster near a small number of salient locations for larger numerosities. Arrays with one dot were an exception as participants tended to fixate near the exact dot location. Differences in fixation patterns across set size suggest that participants were likely to enumerate subsets, rather than objects per se. The fact that fixations were located near the centroid of smaller set sizes suggests individual dots were easily discriminable and perceived as belonging to a single set, whereas larger sets were enumerated as a collection of smaller subsets. 

Similar time required to enumerate a set or the perceived number of subsets

Responses times increased linearly irrespective of whether participants enumerated the number of dots or judged the number of subsets in the same array. This is a significant finding given linear increases in fixation frequency and response times with set size are often taken as evidence that larger numerosities are enumerated incrementally (Schleifer & Landerl, 2011; Sophian & Crosby, 2008; Watson, Maylor, & Bruce, 2007). 

One plausible explanation for the linear association between fixations and response times from both tasks is that participants produce more fixations because they view larger set sizes for longer. Moreover, since arrays with more objects are inherently more complex, they may require a greater perceptual resolution due to crowding or object density (Cavanagh & He, 2011; Li et al., 2010). To examine the likelihood of this explanation, fixation durations were analyzed as a proxy measure of attentional demands of both enumeration tasks. Mean fixation duration was uncorrelated across tasks, with longer fixation durations occurring during subset enumeration than during dot enumeration. This finding suggests enumeration of subsets may involve the integration of visual information over a greater spatial region compared to dot enumeration, and in turn, increases the time required to plan and execute saccades (i.e., greater spatial uncertainty of dot locations, or lower spatial resolution). 

On average, fixations were closer to dot locations during dot enumeration than subset enumeration. This finding is relatively unsurprising given the greater attention required to report the exact number of dots rather than to judge the perceived number of sets. Nevertheless, the similarity in fixation distributions between tasks was constant across set size, which suggests that participants tended to fixate comparable regions of arrays regardless of specific task demands. 

A physiologically plausible model of subset enumeration based on visual salience

Image filtering analyses showed variability in the number of subsets perceived was associated with estimated visual salience of dot arrays. The best model of the behavioral data suggested that participants perceived dots within 2°–2.5° visual angle as belonging to the same subset. Our modeling approach supports earlier behavioral research that proposed numerical judgments involve grouping based on proximity cues (Compton & Logan, 1993; Logan, 1996; Van Oeffelen & Vos, 1982, 1983) that are invariant across affine transformations (Compton & Logan, 1999). Neural substrates have been identified in early human visual cortex that instantiate similar principles of grouping by proximity (Han, Ding, & Song, 2002; Han, Song, Ding, Yund, & Woods, 2001). In contrast, point-based grouping (e.g., k means) provides a less plausible explanation since these models require knowledge of precise dot coordinates for computation to converge on an optimal solution. Moreover, given it is unlikely that object coordinates would be known prior to initiating saccades, an unintuitive prediction of these models would be to maintain central fixation to locate individual dots. 

The limits of proximity grouping were estimated in our model by changing the filter width to approximate different receptive field sizes. Recordings from nonhuman primates have identified neuronal populations in parietal cortex tuned to preferred numerosities; however, the majority of single neurons show no numerosity selectivity (Nieder, 2016). These populations are reported to have relatively large visual receptive fields (Nieder & Miller, 2004; Tudusciuc & Nieder, 2007), and it is possible such neurons are tuned to numerosities presented at locations across distinct receptive fields. Human parietal cortex shows analogous topographical organization of numerosity-selective responses (Harvey et al., 2013), and similar neural properties emerge from artificial neural network models (Stoianov & Zorzi, 2012). Since eye movements are often recorded in nonhuman primate research, it would be interesting for future research to more directly investigate how receptive field sizes and neural activity in eye movement circuitry relate to enumeration of subsets. 

Our findings regarding the specificity of eye movements for enumeration are also consistent with recent evidence suggesting numerosity is encoded by a dedicated neural mechanism or mechanisms (Anobile, Cicchini, & Burr, 2014; Anobile, Turi, Cicchini & Burr, 2015; Burr & Ross, 2008; Cicchini, Anobile, & Burr, 2016; Ross & Burr, 2010) separate from putative mechanisms from processing salient visual spatial features (e.g., area, density, texture; Dakin et al., 2011; Durgin, 2008; Morgan, Raphael, Tibber, & Dakin, 2014; Raphael & Morgan, 2015). The current study contributes new insights into the mechanisms involved in enumerating precise numerosities, as distinct from approximate numerosity discrimination or visual adaptation measured in these other studies. 

Limitations and directions for future research

The high shared variance in response times between the dot and subset enumeration tasks points to a shared visual process. Further research is needed to investigate how visual features such as dot density, eccentricity, and visibility affect the perception of groups, and in turn, whether this affects the speed and accuracy of enumeration. Recent behavioral research has begun to manipulate these visual features by examining “groupitizing” (Starkey & McCandliss, 2014), or the rapid counting of easily discriminable sets of small numerosity (described elsewhere as “groupability”: Van Oeffelen & Vos, 1982, 1984; and counting strategies: Camos, 2003). It would be insightful to use eye tracking to test whether improvements in enumeration via “groupitizing” are a consequence of reduced uncertainty about object coordinates or a greater opportunity for combinatorial strategies involving subsets. Such simple tasks would also be easily adaptable to nonhuman primate protocols (Nieder, 2016). 

The tendency for initial saccades to be directed towards the upper visual hemifield, near the center of arrays may reflect neural encoding of the probability of subset structures in groups of objects. Indeed, it is not apparent that fixating any spatial location is optimal for determining the numerosity of a set, especially given uncertainty about the precise coordinates of individual dots. This initial saccadic tendency may be attributable to the use of a central fixation cue at the beginning of each trial. Van Oeffelen and Vos (1984) reported initial fixations tend to be located near a fixation cue located in the upper left corner of displays, which often resulted in clockwise scan paths. This scanning strategy may be a visual processing heuristic associated with simply accounting for the objects in an array. Changing cue location might influence initial saccades, or alter how visual arrays are scanned and, in turn, affect the latency and accuracy of enumeration. 

Our visual saliency modeling approach could be extended in several ways. Mapping spatial receptive field sizes and deriving visual salience sensitivity functions for each individual would help characterize mechanisms related to the enumeration of subsets. Including orientation filters would also likely capture midlevel image statistics—for instance, grouping based on horizontal or vertical alignment of dots. The visual saliency modeling could also be extended to analyze second-order spatial statistics; in particular, accounting for the potential influence of time-dependent spatial clustering beyond the first-order spatial distribution of fixation densities based on image saliency (Engbert, Trukenbrod, Barthelmé, & Wichmann, 2014). 

Pairing eye tracking with electrophysiology could further substantiate our findings and help characterize the temporal sequence of enumeration. Targeted cortical activation via transcranial magnetic stimulation, for instance, at relevant brain regions involved in enumeration (i.e., early visual cortex, posterior parietal sulcus, and frontal eye fields) would show the functional significance of various neural circuits underlying numerosity computation. 

Conclusions

The current research demonstrates that eye movements during enumeration are affected by the spatial configuration of objects to-be-counted, and the specific task demands. In particular, spatial fixation density analyses combined with image filtering techniques indicate spatial grouping mechanisms related to visual saliency and proximity cues are important determinants of the distribution of enumeration eye movements. Future research could extend these findings by dissociating shifts in attention from overt saccadic eye movements per se, as well as identifying neurophysiological indices of set-based encoding and computation of numerosity. 

Acknowledgments

Commercial relationships: none. 

Corresponding author: Jacob Paul. 

Email: . 

Address: Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, Victoria, Australia.