Evidence for sequential reading effects in screening mammography

Abstract

Screening mammography represents a difficult visual task with significant consequences for the women being screened. It is common for radiologists to read screening mammograms in batches of cases, which opens the possibility for sequential effects from a variety of sources including adaptation to previous cases. Adaptation has been shown to influence the perception and analysis of mammographic images in controlled experiments with non-radiologists, but it remains uncertain how it manifests in practice. We analyzed mammography readings from an observational study in the Netherlands to evaluate possible sequential effects. The study acquired probability of malignancy (PoM) scores and response timestamps during actual mammographic interpretation for 21 radiologists and over 41,600 cases in the Dutch national screening program. We defined a batch of cases for each reader as those interpreted between breaks of more than 10 minutes in sequential timestamps. This gave an average of 14.3 batches per radiologist, and an average batch length of 138.6 cases. The inferred recall rate (true-positive and false-positive interpretations) averaged across radiologists was 2.4%, identical to the reported recall rate for the Dutch breast-cancer screening program. The median reading time, averaged across radiologists, was 19.4 s per case. To evaluate sequential effects, we tested for systematic changes in PoM scores, recall rates, and reading times as the radiologists progressed through the batches. Nonparametric permutation tests were used to establish significant observations. The first 2-3 cases in the beginning of a batch had higher PoM scores and substantially higher recall rates before settling into relatively consistent behavior near the overall averages. Reading times appeared to decay steadily over the first 15 to 20 cases in a batch. Radiologists thus show clear evidence of sequential reading effects over the course of a batch, consistent with the hypothesis that they are adapting to visual properties of the mammograms.