Repeatability and Reliability of Manual Cone Identification in Adaptive Optics Images of Choroideremia

Abstract

Though adaptive optics scanning light ophthalmoscopy (AOSLO) has enabled visualization of the human cone photoreceptor mosaic, the reliability of quantifying cone metrics has not been determined, especially in images of retinal disease. Here, we assess intra- and inter-grader agreement of manual cone identifications in Choroideremia, an X-linked inherited retinal degeneration.

Using a custom AOSLO, split-detection images of the photoreceptor mosaic from 17 Choroideremia patients were acquired at locations surrounding the fovea and along all four meridians. Images were registered, montaged, and 204 regions of interest (ROIs) were selected. Three experienced graders manually identified cones in each of the ROIs. In addition, Grader 1 manually identified cones in all ROIs a second time at least six months later. True and false positive rates and Dice's coefficient were calculated for each ROI using Grader 1's first identifications as ground truth.

Intra-grader agreement was slightly higher than inter-grader agreement. Repeated measurements from Grader 1 gave a Dice coefficient of 0.873?0.058 with a true and false positive rate of 0.943?0.050 and 0.181?0.095, respectively. In comparison, inter-grader agreement yielded Dice coefficients of 0.850?0.090 and 0.814?0.079 for Graders 2 and 3, respectively (0.876?0.128 and 0.884?0.168, true positive rates; 0.159?0.087 and 0.226?0.109 false positive rates).

Manual cone identifications from AO images of Choroideremia show good intra- and inter-grader agreement. The measured agreement places limitations on expectations for algorithms that automatically identify cones in images of Choroideremia and will be needed for quantifying disease progression at the cellular level.

Footnotes

 Conflicts of interest: JIWM US patent 8226236, AGTC funding. RFC consultant for MCW and Translational Imaging Innovations.

Footnotes

 Funding: NIH R01EY028601, NIH U01EY025477, NIH P30EY001583, Research to Prevent Blindness, Foundation Fighting Blindness, the F. M. Kirby Foundation, and the Paul and Evanina Mackall Foundation Trust.