Automated assessment of grating acuity in infants and toddlers using an eye-tracking system

Abstract

The purpose of this study was to assess the feasibility of testing binocular visual acuity using the automated acuity card procedure (AACP)—a new automated method that uses an eye-tracking system. We included participants aged 5 to 36 months old. Binocular grating acuity was tested using the AACP and Teller Acuity Cards (TACs) with a uniform testing distance (55 cm) in random order. Electronic stimuli in the AACP were similar in size and form to TACII and roughly equivalent to the printed TACII stimuli. The AACP tracked the participant's gaze from the recorded image sequences and automatically determined the grating acuity. Differentiation, correlation, and consistency were compared between the AACP and TACs. Ultimately, 77.11% (155/201) of participants completed both tests. Fewer participants failed the TAC test than the AACP (15 [7.46%] vs. 31 [15.42%]). The average duration of the AACP (median interquartile range [IQR] = 60 [IQR = 41] seconds) was significantly shorter than that of the TAC (median [IQR], 185 [IQR = 66] seconds, p < 0.001). AACP and TAC visual acuities were robustly correlated (r = 0.83, p < 0.001). Bland–Altman plots revealed a mean difference between AACP and TAC visual acuities of 0.10 cycles per degree (cpd; 95% limits of agreement = 7.70 cpd). Both the AACP and TACs indicated improved visual acuity with age progression (both, p < 0.001), with no significant differences between the tests. Electronic stimuli were presented using the AACP yielded clinically useful data on grating acuity in infants and toddlers. AACP performance was comparable to that of TACs, the current clinical gold standard for assessing infant vision regarding testability, reliability, and accuracy.

Introduction

Accurate and reliable visual acuity assessment is the basis for the diagnosis and management of ocular conditions. Early detection of abnormal visual acuity, such as congenital cataracts and strabismus, can help prevent permanent visual impairment. Currently, the gold standard for infant vision assessment is the acuity card procedure (McDonald, Dobson, Sebris, Baitch, Varner, & Teller, 1985), which uses the preferential looking test (Teller, 1979). The preferential looking test is used for estimating the visual acuity of infants and others who are not capable of communicating verbally and completing a letter/picture matching test. It is based on the observation that infants and toddlers will look at high-contrast grating stimuli rather than a uniform luminance-matched gray field (Lambert & Lyons, 2016; Sturm, Cassel, & Eizenman, 2011). It also requires reinforcement from a tester who maintains the child's attention (Figure 1A). The development and commercial availability of portable and clinically useful versions of the preferential looking test, such as printed testing stimuli (Teller Acuity Cards [TACs]; Vistech Consultants, Inc., Dayton, OH, USA; TACII, Stereo Optical, Inc., Chicago, IL, USA), has ensured the quantitative measurement of normal visual development (Clifford, Haynes, & Dobson, 2005; Dobson & Luna, 1993; Leone, Mitchell, Kifley, Rose, & Sydney Childhood Eye Studies, 2014) and evaluation of vision deficits (Mash & Dobson, 2005; Stein, Kelly, & Weiss, 2014; Verçosa, Carneiro, Verçosa, Girão, Ribeiro, Pessoa, Almeida, Verçosa, & Tartarella, 2017) in infants and toddlers. 

Figure 1.

 

(A) Infant participant undergoing the Teller Acuity Card II test and (B) automated acuity card procedure.

However, a major limitation of the TAC test is that highly trained, experienced, and patient testers are required to obtain reliable measurements (Clifford-Donaldson, Haynes, & Dobson, 2006). Testers must simultaneously attract the infant's attention while judging their visual acuities based on their behavior. Several research groups have developed new methods of assessing visual acuity in infants and toddlers to address the need for an automated method. Some previous studies have reported semi-automated methods, in which the stimuli were generated by a computer (Hathibelagal, Leat, Irving, Nandakumar, & Eizenman, 2015; Shin, Lee, Wee, Lee, & Hwang, 2013), an iPad (Livingstone, Butler, Misanjo, Lok, Middleton, Wilson, Delfin, Kayange, & Hamilton, 2019), or an electronic card (Mohan, Miller, Harvey, Gerhart, Apple, Apple, Smith, Davis, Leonard-Green, Campus, & Dennis, 2016), although a human tester classified the infant’s responses. Jones et al. used a fully automated acuity measuring system called ACTIVE (Jones, Kalwarowsky, Atkinson, Braddick, & Nardini, 2014). Although the ACTIVE system appeared promising, it was only used to evaluate a small number of infants (N = 30). Moreover, the original hardware is no longer commercially available and the software has been maintained/updated only sporadically since. These factors limit the system's clinical application. 

Considering the above limitations, we developed a new automated method for assessing visual acuity in infants and toddlers by using a high-resolution digital display and an eye-tracking system called the automated acuity card procedure (AACP). In contrast to recent research, we used a 1080p camera instead of a remote eye tracker to capture the gaze. This study aimed to determine whether the AACP yields clinically useful data in the assessment of binocular visual acuity in infants and toddlers. We hypothesized that the testability and test duration of the AACP would be similar to those of the TACII. We also hypothesized that the visual acuity obtained with the AACP would be comparable to that obtained by the TACII. 

Methods

Participants

Participants were infants and toddlers aged 5 to 36 months old (based on gestational age). The eligibility criteria were as follows: (1) gestational age between 37 and 42 weeks at birth; (2) birth weight between 2500 g and 4000 g; (3) normal development according to both the parental report and observation; and (4) no known visual problems or medical conditions. Participants were recruited via advertisements in the Beijing Xicheng Maternal and Child Health Hospital, Beijing Haidian Maternal and Child Health Hospital, and Hebei Tangxian Maternal and Child Health Hospital. Before the test, informed consent was obtained from the parent/guardian of each participant. This research was reviewed by an independent ethical review board and conformed with the principles and applicable guidelines for the protection of human participants in biomedical research. 

Apparatus

The equipment required for the AACP procedure included three parts: a display system, recording system, and an analysis system. We used a 28-inch liquid-crystal display monitor (Samsung UE590 ultrahigh-definition monitor, 62.208 × 34.992 cm screen, 3840 × 2160 pixels, 0.162 mm/pixel, 30 Hz frame rate; Samsung Electronics Co., Ltd., Seoul, South Korea) with an integrated graphics card for the display system. The stimuli for the AACP procedure were square-wave gratings. During each trial, a stimulus (12 cm × 12 cm) with a spatial frequency of different cycles/degree (cpd) and a contrast of 1.00 was used. The stimulus was presented at the center of the left or right sections of the screen, and the remaining sections showed a uniform luminance-matched gray field, similar to the TACII. Stimuli were also roughly equivalent (due to pixel size limits) to those on the printed TACII cards (16 cards, ½ octave steps). We used gamma correction on the monitor; luminance was 1 cd/m², 200 cd/m², and 400 cd/m² for black, mid-gray, and white, respectively. The screen luminance was linearized by an 8-bit look-up table. We used a Logitech C920 high-definition pro webcam (Logitech International S.A., Lausanne, Switzerland) for the recording system with a resolution of 1080p/30 fps, which was located in the center directly below the screen and could be adjusted as needed. During the AACP procedure, the webcam recorded the image sequences of the participant's performance, and each image was cropped to 1280 × 720 pixels. We obtained a timestamped webcam image every 200 ms for synchronization with the acuity test. The analysis system included facial-tracking software and a gaze analysis algorithm. The analysis system was developed in the C++ programming language. The facial-tracking software was based on the open source libfacedetection library development (https://github.com/ShiqiYu/libfacedetection). We developed our gaze algorithm and obtained a patent (China National Invention Patent, No. 201910865074.4). We used an Intel CORE i7-6500U processor (Intel Corporation, Santa Clara, CA, USA) to run all the algorithms of the analyzing system. A standard set of TACs (TACII; Stereo Optical, Inc., Chicago, IL, USA) was also used to measure acuity (Leone et al., 2014), which has a grating of vertical black-and-white stripes presented on one side of a gray card. 

Procedure

For the TAC, the test distance was 55 cm for all participants. Participants were on their guardian's lap during the test (see Figure 1A). Initially, the tester attracted the infant’s attention so that the infant looked straight ahead. Then, the tester held a card (the first card was determined by the infant's age). Based on various infant responses, including fixation, pointing, and/or oral expression, the tester made an initial decision as to whether the infant could see the grating. Coarser or finer gratings were then presented and repeated until the tester selected the finest grating that the child could see. The spatial frequencies of the gratings ranged from 0.32 to 26.0 cpd in 0.5-octave steps. This grating indicated the child’s acuity. To confirm, the tester presented the same grating at least twice. Two effective responses were interpreted to indicate that the grating was seen. Once the response stopped, a coarser grating was attempted again to ensure that the infant had not simply lost interest. 

For the AACP, the participant sat in front of the monitor screen with his/her guardian at a distance of 55 cm (Figure 1B). During calibration, a Graphics Interchange Format (GIF) image (2.0736 cm × 2.0736 cm, 128 × 128 pixels, with background music) appeared on the center of the screen against this isoluminant background. Testers instructed the parents to prompt the child to look at the screen to engage his/her attention. The image remained for a minimum of 3 seconds and disappeared when the infant's gaze was caught by the camera focusing on the image. The analysis system determined whether the infant fixated on the image simultaneously. When any gaze coordinates fell within this area, the trial began and ran automatically, without any intervention from the testers. 

As Figure 2A shows, a grating of set spatial frequency was then presented at the center of the left or right sections of the screen and lasted for 1 second. The initial grating was determined by the infant's age as in the TACII test. The spatial frequencies of the gratings ranged from 0.33 to 30.0 cpd in 0.5-octave steps. Then, the blank screen returned and remained for 200 ms. A grating of the same set spatial frequency was again presented randomly at the center of the left or right sections of the screen. The grating of the same set spatial frequency was presented three times. Then, unified software analysis was performed to obtain the results of these three trials. The gaze algorithm tracked the participant's gaze from the recorded image sequences and determined if they were looking at the stimulus. If the gaze remained in a 12 cm × 12 cm square area (the stimulus area) centered on the target continuously for more than 400 ms of the entire stimulus presentation (1 second), the infant was judged to have looked at the stimulus (a “hit”). Otherwise, the trial was scored as a “miss.” 

Figure 2.

 

(A) Hit and miss for each trial. The red dot indicates the gaze point of the participant. The gaze point of the participant entering the green square indicates that the participant has watched the target. (B) The protocol used in the AACP procedure.

The protocol (Figure 2B) used to adapt the spatial frequency of the grating was a 2-up 2-down staircase as follows: 

• (1) If there were at least two hits in the first three trials, the spatial frequency was increased by 0.5 octaves. Thereafter, if there were at least two hits in the three trials, the spatial frequency was increased by 0.5 octaves. If fewer than two hits were presented in the following grating, the current spatial frequency was presented three more times. If there were fewer than two hits, the test was stopped and the spatial frequency of the lower level was output as the acuity threshold. Otherwise, if there were at least two hits in the highest frequency (30 cpd) trials, the test was stopped and the highest frequency was output as the acuity threshold.
• (2) If there was more than one miss in the first three trials, the spatial frequency was presented three more times. If there was more than one miss again, the spatial frequency was decreased by 0.5 octaves. Thereafter, when the first reversal was met, the test was stopped and the current spatial frequency was output as the acuity threshold. Otherwise, if there was more than one miss in the lowest frequency (0.33 cpd) trials, the test was stopped and no result was output.

GIF images were presented after every three trials to maintain interest. If no gaze data were available within 1 minute during the GIF image presentation (e.g. if the infant turned away or closed their eyes), the test was stopped and no result was output. 

Psychophysical procedures used for participants

Two experienced testers performed the tests: while one conducted the AACP, the other conducted the standard acuity card procedure using the TACII. A testing distance of 55 cm was used for all infants. The approximate amount of time taken to position the participant for both tests was 30 seconds. Participants were examined once using each procedure in random order. Infants who were able to cooperate with the first test could proceed to the second test. For both tests, the infant's behavior and attention were recorded. According to the testers’ observations or the AACP test video, if an infant was attentive throughout the test and their results were reliable, the tester recorded the results. Infants who were not alert enough of the time or did not complete the test were considered to have failed the test. The responses of participants who fell asleep, became distressed, turned away, or otherwise did not complete the test were regarded as failed and excluded from further analysis. 

Data analysis

For statistical analyses, acuity data and duration data were transformed to a log₁₀ scale. The Shapiro–Wilk test and histogram visualization were used to assess the normality of the distribution. The log-transformed data were normally distributed. The data analysis included the calculation of the 95% limits of agreement between the visual acuities of the automated and nonautomated acuity card procedures. The paired t-test was used to compare the differences between these two procedures when appropriate. Between-group comparisons were performed using one-way analysis of variance. Preliminary correlation was determined based on the Pearson correlation coefficient. To further test the relationship, we used linear regression. Bland–Altman assessment was used to calculate the agreement between the AACP and TACs. Numerical data are expressed as the mean ± standard deviation. Non-normal data are expressed as the median (interquartile range, IQR). Any p values < 0.05 were considered statistically significant. 

Results

Testability and test duration

The final sample included 201 participants (46.77% female children; age range = 5–36 months). Altogether, 155 (77.11% of 201) participants completed both tests. Fifteen participants (7.46%) failed the TAC test, which was a lower percentage than the 31 participants (15.42%) who failed the AACP. Figure 3 shows the results of participants who completed both tests. 

Figure 3.

 

Final results of 155 participants who completed both the AACP and the TAC. The abscissa and ordinate values indicate each participant's AACP and TAC grating acuities, respectively. AACP, automated acuity card procedure; TAC, teller acuity card.

As Figure 4 shows, there was a significant difference between the duration of the automated (median [IQR], 60 [IQR=41] seconds) and nonautomated (median [IQR], 185 [IQR = 66] seconds) acuity card procedures (paired t-test of log-transformed data; t₉₂ = −23.24, p < 0.001). The TAC test duration was approximately three times higher than that of the AACP. The duration times represent the time from when the participants sat down to test completion and do not include the setup time for either test, that is, seating the child at the appropriate distance. 

Figure 4.

 

Box-plot of test durations (n = 155) using the TAC and AACP procedures. Duration represents the time from when participants sat down to test completion. AACP, automated acuity card procedure; TAC, teller acuity card.

Measured acuity

The distributions of acuities determined using the AACP or TACII are shown in the Table 1. There was a significant correlation between the visual acuities yielded by the AACP and TAC (r₁₅₃ = 0.83, p < 0.001). The visual acuity of AACP predicted that of TAC with significant accuracy (β = 0.827, t = 18.226, p < 0.001, 95% confidence interval = 0.584–0.725). The agreement between the AACP and TACs is shown in Figure 5. The mean difference (AACP-TAC) was 0.10 cpd, and the 95% limits of agreement were 7.70 cpd. Nine (5.81% of 155) subjects were outside the 95% limits of agreement. A total of 76.77% (119/155) of acuity measures were within less than or equal to 0.5 octaves of the AACP and TACs. A total of 89.68% (139/155) of acuity measures were within less than or equal to 1 octave of each other. 

Table 1.

 

Average AACP and TAC grating acuities according to age. AACP, automated acuity card procedure; IQR, interquartile range; TAC, teller acuity card. Paired t-test of log-transformed data. One way ANOVA test of log-transformed data.

 

Figure 5.

 

Bland–Altman plot showing the agreement between AACP and TAC acuities for participants. The difference between the acuities is plotted against the mean of the acuities. The dotted lines indicate the 95% limits of agreement. AACP, automated acuity card procedure; TAC, teller acuity card.

There were significant correlations between acuity and age (r₁₅₃ = 0.59 and 0.71, p < 0.001, for AACP and TACs, respectively). The average acuities of both the AACP and TACs for age are plotted as a function of age in Figure 6, together with the normative data acquired previously using TACs (Salomão & Ventura, 1995) and TACII (Leone et al., 2014). Median acuity improved with the progression of age. After controlling for age-related variations, no significant differences in acuities (measured using either the AACP or TAC test) were noted (see the Table 1). 

Figure 6.

 

Scatterplots of average grating acuities plotted against age. The dashed-dotted line with a black star is the mean acuity for the automated acuity card procedure (AACP), and the dashed-dotted line with the gray star is the Teller Acuity Card (TAC) mean acuity. The dotted lines are in the upper and lower 95% range as previously described by Leone et al. (2014). The solid lines are in the upper and lower 95% range as previously described by Salomão and Ventura (1995).

Discussion

This study demonstrates that the AACP, consisting of display, recording, and analysis systems, can determine grating acuity in infants and toddlers with accuracy comparable to that of the current gold standard method. The AACP is conceptually similar to the acuity card procedure; however, it is an automatic procedure and does not require any intervention from the testers. It offers several advantages over the TACII: it allows the electronic adjustment of spatial frequency, which means that there is no longer a need to print cards; different types of stimuli can be tested easily; and it is more time-efficient. Furthermore, it allows the maintenance of constant luminance across testing environments and uses a constant algorithm throughout the test; consequently, the accuracy and objectivity of the results can be guaranteed. 

Test failure

In this study, 22.88% of the children failed to complete both tests. Furthermore, 15 (7.46%) children failed the TAC. During this study, the average time required for a child to finish the acuity card procedure was 3 minutes, which is similar to that reported in previous studies (Qiu, Li, & Yan, 2011; Teller, McDonald, Preston, Sebris, & Dobson, 1986). The duration of TAC was much longer than that of AACP. As previous studies have demonstrated, infants often exhibit high interindividual variability and high levels of inattentiveness (Jones, Kalwarowsky, Braddick, Atkinson, & Nardini, 2015; Werner & Marean, 1991). In our study, nearly half of the infants (7/15) who did not complete the test were 5 to 6 months old. One possibility is that some younger infants did not participate long enough to achieve true threshold acuity. Conversely, the reason for older infants to fail the acuity card procedure may have been distress and unwillingness to cooperate. In future studies, breaks should be taken as required to avoid uncooperative behavior. 

In total, 31 (15.42%) children failed the AACP. Over one-third of infants (12/31) who did not complete the test were 5 to 6 months old. One possibility is that a 1-second trial may be too short for younger infants to distinguish the grating. Longer durations may be required to avoid false negative results when testing younger infants. For older toddlers, although the AACP displayed GIF images and music to attract their gaze directly to the center of the screen before presenting the stimulus, we observed that this strategy sometimes failed to attract their attention. If infants turned away or moved outside the trackable area, trials were automatically paused as they failed to determine the true visual acuity. When testing infants older than 20 months, the initial frequency was 2.4 cpd, although their average acuity was 10 cpd, and they tended to lose attention during the tests. The location of the stimulus had only two options: either at the center of the left or right sections of the screen. The test had a chance of false positive results caused by random gazing but which were recognized by the system, leading these infants to receive higher scores than they deserved. Considering infants’ limited patience, they were only examined once in this research. In further research, observing the infants’ behavior and repeating the tests as required might help overcome these issues. Conversely, a component of the TAC procedure is the inclusion of a tester who maintains the child's attention and makes a judgment. Therefore, future studies should focus on developing methods to improve the engagement of a child's attention and decrease the system's miscalculation. For example, randomly varying the target location between trials may decrease the chances of false positive responses (Jones et al., 2014). Such methods could potentially offer a more objective way to assess behavior and reduce reliance on tester training and experience, which may improve the feasibility of acuity testing and facilitate its widespread use in clinical and research settings. 

Acuity estimate

Regarding accuracy, the acuity estimates yielded by the AACP were correlated with those of the acuity card procedure within subjects. However, our study showed that the age data differed between the two procedures. Possible reasons for this are as follows. First, contrast and luminance differences may have contributed to differences in scores between the two tests. Second, design-related differences may have led to differences in the tasks between the two tests. For the TAC/TACII tests (Clifford-Donaldson et al., 2006; Dobson & Luna, 1993), effective results rely on a subjective assessment of the infant's or toddler's looking behavior by an experienced clinician. For the AACP, the analyzing system attracts the infant or toddler's gaze and detects the grating directly. Finally, the number of participants included in this study was small. In future studies, many participants must be included to obtain more data on age norms. The present study demonstrates the validity of the AACP; however, a larger sample is required to provide normative data. 

Limitations

In this study, the 1-second trial in AACP may be too short for younger infants to distinguish the grating. Longer durations may be required in the future to avoid false negative results when testing younger infants. The stimuli for the AACP procedure were square-wave gratings, such as the TAC, which has “edge effects,” increasing the chance of false positive results in older children and adults. To avoid these effects in the future, it should be made a Gabor. The location of the stimulus had only two options: at the center of the left section or at the center of the right section of the screen. The protocol should be updated so that the target location varies between trials and include an “invisible target” trial at the end of the test to indicate the guessing rate. 

In this study, participants were healthy infants and were only examined once. We also required repeat and abnormal infant data to assess the clinical applicability of the AACP. We did not measure monocular visual acuity, which is clinically important. Although it was not demonstrated in this study, it should be noted that the AACP can also track a single eye and measure monocular visual acuity. Based on previously published data (Qiu et al., 2011; Salomão & Ventura, 1995), we expected monocular acuity thresholds to be lower than those of binocular acuity thresholds in this study. 

Conclusions

We have shown that the AACP is a valid measure of grating acuity in infants and toddlers and has good testability and reliability. The test duration of the AACP was much shorter than that of the TACII due to its gaze-tracking ability and fully automated system. In terms of accuracy, we demonstrated that the performance of the AACP was comparable to that of the acuity card procedure. Testing of binocular visual acuity in infants and toddlers using the AACP yields clinically useful data. Further research is needed to develop an improved method for engaging the infants’ attention and decreasing the system's miscalculations. Furthermore, we are in the process of obtaining additional data on age norms and collecting monocular data using the AACP, and these results will be reported in the future. 

Acknowledgments

Supported by Capital's Fund for Health Improvement and Research (No. 2018-2Z-4076) and Peking University First Hospital SEED Research Funds (2019SF31). 

Commercial relationships: J. Wen, None; B. Yang, (P); X. Li, (P); J. Cui, (P); L. Wang, (P). 

Corresponding authors: Xiaoqing Li, Jinshi Cui, Li Wang. 

Emails: [e-mail], [e-mail], [e-mail]. 

Addresses: Department of Pediatric Ophthalmology, Peking University First Hospital, No. 8 Xishiku Street, Xicheng District, 100034 Beijing, China; Key Laboratory of Machine Perception, Peking University, No. 5 Yiheyuan Lu, Haidian District, 100871 Beijing, China; School of Psychological and Cognitive Sciences, Peking University, No. 5 Yiheyuan Lu, Haidian District, 100871 Beijing, China.