Abstract
For any task, individuals will differ from each other in initial performance, learning rate, and ultimate task competence. As such, one question is how well initial task performance relates to eventual competence? Do those who start out as the better performers remain so later? Practically, this question is especially important for visual search performance given how many professions rely on successful search (aviation security, radiology, etc.). Our previous work (Ericson, Kravitz, & Mitroff, 2017) showed the minimal unit of data (participants' response time on the first trial) could predict later success. However, while it was theoretically interesting to examine the smallest amount of data needed to predict later levels of performance, it is practically interesting to maximize classification accuracy of eventual success from initial performance. In this study we leveraged "big data" (>3.3 billion trials from >14 million unique devices) from the Airport Scanner mobile app game (Kedlin Co.) and linear classification techniques to expand beyond single trial classification methods. Improved prediction of highest rank achieved in the visual search based game, our proxy for ultimate visual search competence, was achieved here by: 1) examining performance metrics in addition to response time (e.g., hits and false alarms), and 2) reducing trial-by-trial variability in performance metrics across participants by controlling for factors that affect trial difficulty (e.g., number of distractors, average difficulty for specific targets, and other in-game factors). We will present data that demonstrate it is possible to use a small sample of search task performance from early in learning to predict later success with high accuracy. The data highlight that while everyone can improve at a task with experience, those who start out the best tended to remain the best. Morever, it is possible to use the initial data to classify later top and bottom performers.
Meeting abstract presented at VSS 2018