Abstract
Greene, Liu and Wolfe reported a negative result for identifying task using eye movements (2012). Others have challenged this finding, often by substituting dynamic for static features (Boisvert & Bruce, 2016). We propose instead mapping eye movements onto a latent dynamic manifold with connected, overlapping regions of related task effects. This manifold’s structure is defined using features extracted from a variational auto-encoder trained on multi-dimensional time-series of values for two different salience maps at observer fixation locations (GBVS: Harel et al., 2007; modified semantic similarity: Rose & Bex, 2018). This approach helps reconcile Greene’s findings with others’ by treating eye movement as less “task driven” than “tasky”: affected by current task while remaining influenced by those performed in the past. We therefore hypothesized that observers’ sequential performance of different tasks–as in Greene and colleagues’ study–may deflect behavior across manifold regions, decreasing classifiability. Performing the same task should result in more regular trajectories within a manifold, increasing classifiability. We tested this hypothesis on three data-sets first by comparing classification accuracy per task/trial using twenty-five manifold features and a SVM classifier. Second, we compared sequential determinism and entropy in trial-to-trial sequences of feature distances to separating hyperplanes using multidimensional recurrence quantification analysis (Wallot, 2016). For two sets (Koehler et al.,2014; Borji&Itti, 2014,Expt.2), subjects always performed the same task. In the third (created in-lab), tasks were randomized across trials. Classification performance was significantly above chance and equivalent to that reported elsewhere for random order studies (Borji & Itti, 2014:24.2%, ours:24.6%, chance:14.2%; Boisvert & Bruce,2016:53.4%; ours:51%, chance:33.3%) but not the third (ours:39.5%, chance:33.3%). Determinism was also significantly higher and entropy significantly lower for the random task order experiment than for the others (Kruskal-Wallis, p< 0.001). Our method therefore both retains high discrete classification peformance while connecting task effects to other efforts concerning serial dependencies (Fischer & Whitney,2014).