October 2020
Volume 20, Issue 11
Open Access
Vision Sciences Society Annual Meeting Abstract  |   October 2020
What you don’t see can help you: Image triage in human-AI interactions
Author Affiliations & Notes
  • Chia-Chien Wu
    Harvard Medical School; Brigham and Women's Hospital
  • Leah Kumle
    Goethe University Frankfurt
  • Makaela Nartker
    Johns Hopkins University
  • Jeremy Wolfe
    Harvard Medical School; Brigham and Women's Hospital
  • Footnotes
    Acknowledgements  NIH EY017001, CA207490
Journal of Vision October 2020, Vol.20, 519. doi:https://doi.org/10.1167/jov.20.11.519
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Chia-Chien Wu, Leah Kumle, Makaela Nartker, Jeremy Wolfe; What you don’t see can help you: Image triage in human-AI interactions. Journal of Vision 2020;20(11):519. https://doi.org/10.1167/jov.20.11.519.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

The performance of artificial intelligence (AI) has reached expert levels for several medical image screening tasks. Nevertheless, in clinical settings, the combination of AI and expert radiologists often fails to produce better outcomes than either working alone. We gave non-expert observers simulated AI assistance to search for target textures in arrays of eight color textures located on a virtual circle. Target colors were drawn from a distribution that differed by 2.2 standard deviations from the distractor distribution. Four conditions were tested: 1) “On-demand AI” gave a probability that specific items were targets only when asked by the observer. 2) “Image triage” used a liberal criterion to filter the images that were shown to the observer. If AI was “sure” that an item was not a target, it was not shown, reducing the set size. If AI eliminated all items, the entire trial was not shown. 3) “Both”: First, images were triaged. Observers could then query remaining stimuli for the probabilities that they were targets. 4) “Control” condition had no simulated AI. Prevalence was 50% or 10% in different blocks. Results: Triage improved performance. On-demand AI did not. Prevalence did not change this pattern, though miss errors were elevated at low prevalence. In Experiment 2, On-demand was replaced by a “Second Reader” AI that gave its opinion after humans’ response and allowed humans to change their previous response. Again, Triage helped. Second Reader AI did not. Unlike Wolfe and Nartker’s (VSS2019) results, using Both (AI for triage and second reader) was not better than Triage alone, perhaps because the Second Reader was not effective in this paradigm. Similar results were obtained at low prevalence. In search tasks, Image Triage, where AI saves time by eliminating some stimuli, may be more promising than methods where AI tries to offer positive advice.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.