Large scale validation of an early-age eye-tracking biomarker of an autism spectrum disorder subtype
(Full-text capture 2026-09-21; web artifacts lightly stripped; truncated.)
Teresa H Wen
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
1,✉, Amanda Cheng
Amanda Cheng
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
Charlene Andreason
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
1, Javad Zahiri
Javad Zahiri
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
1, Yaqiong Xiao
Yaqiong Xiao
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
1, Ronghui Xu
Ronghui Xu
3Herbert Wertheim School of Public Health and Department of Mathematics, University of California, San Diego, La Jolla, CA USA
3, Bokan Bao
Bokan Bao
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
2Department of Bioinformatics and Systems Biology, University of California, San Diego, La Jolla, CA USA
1,2, Eric Courchesne
Eric Courchesne
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
Cynthia Carter Barnes
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
Steven J Arias
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
1, Karen Pierce
Karen Pierce
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
1,✉
- Author information
- Article notes
- Copyright and License information
1Autism Center of Excellence, Department of Neurosciences, University of California, San Diego, 8110 La Jolla Shores Dr., La Jolla, CA 92037 USA
2Department of Bioinformatics and Systems Biology, University of California, San Diego, La Jolla, CA USA
3Herbert Wertheim School of Public Health and Department of Mathematics, University of California, San Diego, La Jolla, CA USA
✉
Corresponding author.
Received 2021 Aug 10; Accepted 2022 Feb 28; Collection date 2022.
© The Author(s) 2022
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
PMCID: PMC8917231 PMID: 35277549
Abstract
Few clinically validated biomarkers of ASD exist which can rapidly, accurately, and objectively identify autism during the first years of life and be used to support optimized treatment outcomes and advances in precision medicine. As such, the goal of the present study was to leverage both simple and computationally-advanced approaches to validate an eye-tracking measure of social attention preference, the GeoPref Test, among 1,863 ASD, delayed, or typical toddlers (12–48 months) referred from the community or general population via a primary care universal screening program. Toddlers participated in diagnostic and psychometric evaluations and the GeoPref Test: a 1-min movie containing side-by-side dynamic social and geometric images. Following testing, diagnosis was denoted as ASD, ASD features, LD, GDD, Other, typical sibling of ASD proband, or typical. Relative to other diagnostic groups, ASD toddlers exhibited the highest levels of visual attention towards geometric images and those with especially high fixation levels exhibited poor clinical profiles. Using the 69% fixation threshold, the GeoPref Test had 98% specificity, 17% sensitivity, 81% PPV, and 65% NPV. Sensitivity increased to 33% when saccades were included, with comparable validity across sex, ethnicity, or race. The GeoPref Test was also highly reliable up to 24 months following the initial test. Finally, fixation levels among twins concordant for ASD were significantly correlated, indicating that GeoPref Test performance may be genetically driven. As the GeoPref Test yields few false positives (~ 2%) and is equally valid across demographic categories, the current findings highlight the ability of the GeoPref Test to rapidly and accurately detect autism before the 2nd birthday in a subset of children and serve as a biomarker for a unique ASD subtype in clinical trials.
Subject terms: Autism spectrum disorders, Diagnostic markers, Paediatric research
Autism spectrum disorder (ASD) begins during prenatal life1, 2, yet most children do not receive a diagnosis and start treatment until 3–4 years later3, 4. Although genetic, neural, metabolomic, and molecular systems are adversely impacted in ASD2, 5, 6, it is nevertheless detected and diagnosed using clinical judgement.
There has been a recent surge in research designed to discover biologically-based markers of ASD which can increase the pace of diagnosis, remove the requirement for highly-trained professionals, provide prognostic information, guide treatment plans, or be used as outcome measures in clinical trials7. Currently, only two ASD biomarkers are being considered for the FDA Biomarker Qualification Program8, 9. However, they were established at “late” ages in children and therefore may not be generalizable to toddlers and infants, for whom biomarkers are of greatest utility. Moreover, these biomarkers only identify a subset of ASD children, indicating that additional biomarkers for other ASD subtypes are needed.
Dramatically reduced attention to social information is a key feature of ASD noted since its discovery in 194310. Unsurprisingly, considerable effort has been leveraged to understand and quantify social visual attention abnormalities, most recently using eye-tracking11– 29. Despite varying stimuli and participant age, a meta-analysis of 38 published eye-tracking studies indicated that subjects with ASD attend definitively less to social stimuli compared to typically developing individuals30.
Our previous eye-tracking work using a novel preferential looking paradigm known as the ‘GeoPref Test’ identified a subgroup of ASD toddlers with heightened visual attention towards geometric relative to social images16, 25. The effect was robust with > 85% test–retest reliability, 86–100% positive predictive value (PPV), and 98% specificity, although sensitivity was consistently lower at ~ 20%16, 25. ASD toddlers with a strong non-social preference also had higher symptom severity, worse language and cognitive ability16, weak functional connectivity between social-visual brain networks31, and worse school-age outcomes compared to social-preferring toddlers20. Because this work was conducted within the context of population-based screening32, 33, one strength was the inclusion of non-ASD contrast groups which mimic the natural constituency found during routine pediatric practice. This is of considerable importance for the development of biomarkers with real-world utility. Weaknesses of the initial studies, however, include small sample sizes (i.e., 110 and 333), and a lack of rigorous methodology to examine predictive validity. We also failed to combine metrics to improve predictive accuracy, focusing instead on fixation levels alone. While other groups have utilized similar paradigms13– 15, 26, 27, 34– 36 or a near identical version of the GeoPref Test25 and report similar results19, 37– 48, validation statistics are often unreported.
In addition to validation, establishing impactful biomarkers requires understanding for whom, and at what ages, the marker best applies. In fields outside of autism it is well known that biomarker efficacy differs by race, ethnicity, age, and sex49, 50. With minor exception, eye-tracking studies within the autism field often have relatively few subjects11, 22, 29, 51, yet sample sizes in the thousands are necessary for resolving demographic effects and establishing medical biomarkers52– 54. Moreover, there is a need to examine associations between biomarkers and clinical profiles as a pathway towards individualized medicine. Finally, ASD biomarkers which are tuned for high specificity/low false positive rate are necessary for circumventing the financial burden and familial stress associated with false positive results.
There is also evidence for a genetic component of visual social preference55– 58, but few studies have explored this using eye-tracking. We addressed this previously by comparing GeoPref Test performance in sibling pairs and found that among sibling pairs concordant for ASD, fixation to geometric images was strongly correlated16. In a different study, the time non-ASD monozygotic twins spent looking at eyes or mouth in an eye-tracking task was highly intercorrelated59. Such findings underscore the potential genetic basis of social visual attention patterns.
Given the necessity for well-developed, clinically relevant ASD biomarkers both for diagnostic purposes and use in clinical trials, the goal of the present study was to comprehensively validate an eye-tracking based biomarker in a large, diverse group of toddlers, producing the largest eye-tracking study of ASD and other delays to date. Here, we report ASD classification accuracy using both a simple, scalable approach associated with a single metric—percent fixation—that can be easily leveraged by non-researchers, as well as a more complex approach which utilizes machine learning algorithms to conduct tenfold validation and an independent replication set based on multiple eye-tracking metrics.
Methods
Participants
Subjects were referred through the community or via a population-based screening method known as Get SET Early 32, 33. Following screening at well baby check-ups using the CSBS IT-Checklist60, toddlers were referred to the University of California, San Diego Autism Center of Excellence for in-depth diagnostic evaluations and eye-tracking, and invited for repeat testing every ~ 12 months until age 3. Toddlers were assessed by licensed Ph.D.-level clinical psychologists blind to eye-tracking results using the Mullen Scales of Early Learning61, the Autism Diagnostic Observation Schedule62, and the Vineland Adaptive Behavior Scales63. Parents were given diagnostic feedback and toddlers referred for treatment as appropriate.
Of the 1,685 toddlers enrolled in the study, 266 (15.8%) were excluded largely due to compliance (See Supplemental Methods eFigure 1). The remaining 1,419 toddlers (mean age: 24.37 months, range: 12.00–49.11) were separated into diagnostic groups based on most recent diagnoses including ASD, ASD features (ASD-Feat), global developmental delay (GDD), language delay (LD), typically developing (TD), typical toddlers with an ASD sibling (TypSibASD), and Other (Table 1 and Supplemental Methods). This final sample is independent from our previous work16, 25. To validate the GeoPref Test on the largest sample possible, secondary analyses included 444 toddlers from our previous work resulting in a cumulative sample of 1,863 toddlers. Among this sample are 11 monozygotic twins, 27 dizygotic twins, and 109 sibling pairs. Given the goal of early biomarker discovery, the first (i.e., youngest age) eye-tracking data collection was used in analyses.
Table 1.
Summary of clinical characteristics for 1,863 toddlers who completed The GeoPref Test. Values are presented as means (standard deviations are noted in parentheses unless otherwise noted, and age range provided in brackets). ASD: Autism Spectrum Disorders, ASD-Feat: ASD Features, GDD: Global Developmental Delay, LD: Language Delay, TD: Typically Developing, TypSibASD: Typical Sibling of subject with ASD; ADOS: Autism Diagnostic Observation Scale, SA/CoSo Score: Social Affect/Communication Social Score, RRB Score: Restricted and Repetitive Behavior Score.
| Mean (SD) | |
|---|---|
| ASD (N = 725) | ASD-Feat (N = 103) |
| :— | :— |
| M/F | 563/162 |
| Age, months | 26.40 (8.25) |
| Range | [12–49] |
| Ethnicity (%) | |
| Hispanic or Latino | 32.69% |
| Non-Hispanic or Latino | 56.00% |
| Unknown/not reported | 11.31% |
| Race (%) | |
| Caucasian | 52.14% |
| American Indian/Alaska Native | 0.83% |
| Asian | 13.10% |
| Black/African American | 2.48% |
| Pacific Islander/Native Hawaiian | 0.83% |
| More than one race | 9.79% |
| Not reported/unknown/other | 20.83% |
| Mullen (t score) | |
| Visual reception | 38.00 (13.04) |
| Fine motor | 37.59 (16.23) |
| Receptive language | 28.47 (12.78) |
| Expressive language | 27.80 (13.03) |
| Early learning composite | 69.01 (19.39) |
| Vineland (standard score) | |
| Communication | 77.20 (13.90) |
| Daily living | 84.65 (12.61) |
| Socialization | 84.23 (11.97) |
| Motor skills | 90.76 (10.67) |
| Adaptive behavior composite | 81.08 (11.12) |
| ADOS (module T or 1) | |
| ADOS RRB score | 4.36 (2.16) |
| ADOS Sa/CoSo tot score | 13.24 (4.78) |
| ADOS total score | 17.60 (5.87) |
Eye-tracking apparatus, stimuli, and procedures
Eye-gaze data was collected using the Tobii T120 (Tobii, Stockholm, Sweden; www.tobii.com; 60 Hz sampling rate; 1280 × 1024) while toddlers watched ‘The GeoPref Test’ (62.22 s), which consisted of two rectangular areas of interest (AOIs, 525 × 363 pixels) each containing dynamic geometric (DGI) or social images (DSI; social images used with permission from Gaiam Americas Inc., Copyright 2003, Gaiam Americas, Inc.), identical to stimuli used in our previous work16, 17, 25 (Fig. 1 A; Supplemental Methods eFigure 2). To control for biases due to spatial location, side of stimulus presentation varied across subjects.
Figure 1.
Validation of visual attention patterns among a combined sample of 1,863 toddlers of varying developmental types. ( a) Sample images from the GeoPref eye-tracking test across the first 12 s. Social images courtesy of Gaiam Americas Inc., Copyright 2003, Gaiam Americas, Inc. To standardize initial fixations across toddlers, testing was preceded by a fixation star which flashed for 6.23 s. The entire length of the test was 62.23 s. ( b) In the current study, 444 toddlers from a previous analysis16 were combined with a new, independent sample of 1,419 toddlers in order to increase power and provide the largest dataset possible for subsequent analyses. As such, scatterplot demonstrates percent fixation to dynamic geometric images (DGI) or dynamic social images (DSI) for the combined sample of 1,863 toddlers. Percent fixation was defined as fixation duration within geometric or social areas of interest divided by total fixation duration to the entire video. Dashed lines depict threshold for 69% fixation. Green triangles indicate average percent fixation. Notched boxplots show median, range, and first/third quartiles. Notch ranges indicate 95% confidence intervals around the median. TP: true positive, FP: false positive. F statistic pictured was obtained from a one-way ANOVA conducted to compare percent DGI fixation across diagnostic groups. ( c) Effect size matrix of pairwise comparisons of average percent fixation to DGI for toddlers of varying developmental types. Warmer colors indicate larger effect sizes and cooler colors indicate smaller effect sizes. ( d) Validation statistics for classification of toddlers as ASD vs. non-ASD using a 69% fixation threshold after grouping toddlers into 4-month age bins. Toddlers > 40 months were binned together given the small sample size of toddlers beyond this age. ( e) ROC curves showing classification performance (ASD vs. non-ASD) among 20% of toddlers which make up an independent, hold-out test using left, percent DGI fixation alone (threshold: 61.3%), or right, multiple eye-tracking metrics (percent fixation, threshold: 61.3% and saccades/second, threshold: 2.29 saccades/second). ASD: Autism Spectrum Disorders, ASD-Feat: ASD Features, GDD: Global Developmental Delay, LD: Language Delay, TD: Typically Developing, TypSibASD: Typical Sibling of subject with ASD, AUC: Area under the curve, Neg. Pred. Val.: Negative Predictive Value, Pos. Pred. Val.: Positive Predictive Value.
To ensure that only the toddler’s gaze was tracked and free from parent influence, standardized instructions were read to parents. A five-point calibration was then performed using animated cartoon ducks with sounds, and data was only used if calibration results, determined via graphical output and verified via screenshots, fell within manufacturer-reported parameters (accuracy, 0.5 degrees64). For a subset of toddlers, a flashing star with chime appeared for 6.23 s prior to the start of the experiment to ensure toddlers fixated to the screen.
Data was processed using Tobii Studio (Tobii Fixation Filter, velocity threshold: 35 ms/window). Total fixation duration, fixation count within each AOI, and fixation duration within each AOI were exported and analyzed offline. Percent fixation duration/AOI was computed by dividing the total fixation duration within an AOI by the fixation duration across the entire video. N-1 total fixations/total fixation duration was used to calculate saccades/sec within each AOI.
Defining ASD subgroups
For comparability with our earlier work and to tune the test towards a low false positive rate, 69% fixation to either DGI or DSI was used to distinguish between ASD toddlers who strongly preferred to visually fixate on geometric images (ASDGeo) or social images (ASDSoc)16, 25. Given that experience-dependent mechanisms may differentially impact toddlers with distinct attention patterns, stratifying ASD toddlers in this way may have relevance for understanding long term outcomes among subtypes identified through eye-tracking. ASD toddlers who lacked a strong preference for either stimulus (i.e. they fixated on DGI for 32–68% of the time) were categorized as ASD Middle responders (ASDMid).
Statistical analyses
All analyses were conducted using R and relevant packages (e.g., pROC, ROCR65– 74). Results are reported using associated 95% confidence intervals (CI) and effect sizes as appropriate.
Visual attention preference stratified by diagnosis and correlation with clinical measures
One-way ANOVA was performed to compare percent fixation towards DGI across diagnostic groups. Follow-up pairwise comparisons of group means were conducted using Tukey’s HSD.
Linear regression with percent DGI fixation as the outcome variable and sex, age, and diagnosis as predictor variables was used to examine associations between demographics and DGI fixation levels. Pearson’s correlations were conducted to examine relationships between DGI fixation and performance on clinical measures.
Across ASD subgroups, performance on clinical measures and associated subscales was assessed via one-way ANCOVA with age and sex as covariates. Subgroup-level differences were assessed using Tukey’s HSD.
Classification accuracy: traditional approaches
To determine whether validation statistics for the current sample were comparable to those observed in our previous work16, 25, and to tune the test in favor of a low false-positive rate, the same 69% fixation threshold was applied here to the full cohort to compute Receiver Operating Characteristic (ROC) curves, an Area Under the Curve (AUC) statistic, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).
Validation statistics were also computed after stratifying toddlers based on age bins (4-months), ethnicity, race, and sex.
Saccade profiles within ASD
Our previous work indicated that ASDGeo toddlers exhibited significantly fewer saccades when viewing DGI and significantly more when viewing DSI relative to other ASD toddlers16, 17, 25. To determine whether these patterns were also present in this larger sample, a one-way ANCOVA with age and sex as covariates and a follow up Tukey’s HSD test was used.
Classification accuracy: cross validation with independent hold out set
Rigor, generalizability, and diagnostic accuracy of the GeoPref Test were examined using a tenfold cross-validation approach conducted on 80% of the cohort who were randomly selected to be included in the analysis. Ten-fold cross-validation is a widely used method for parameter tuning and threshold selection in statistical learning and classification problems75. Average performance statistics for all 10 validations were averaged and reported, and then used to classify the remaining 20% of toddlers which made up an independent, hold-out test set. See Supplemental Methods.
Classification sensitivity enhancement was also examined using tenfold cross-validation following the inclusion of both percent fixation and a second eye-tracking metric, number of saccades per second within DSI, as variables of interest.
Test–retest reliability
Additional eye-tracking sessions (not used in aforementioned analyses) occurred in 535 toddlers (mean age: 19.48 ± 6.18 months) within 1 to 25 + months of their first eye-tracking test. After grouping toddlers based on interval length (immediate: 0–1 month, n = 39; short term: 2–6 months, n = 100; intermediate term: 7–12 months, n = 211; long term: 13–24 months, n = 163; and very long term: > 25 months, n = 22), intraclass correlations and paired t-tests were used to examine differences in percent DGI fixation between tests.
Development of visual social attention preference across the first years of life
To better understand the trajectory of typical versus atypical visual attention patterns, Pearson’s correlations comparing age and percent DSI/DGI fixation were conducted for all diagnostic groups. The impact of age on eye-tracking performance was further examined using linear regression with percent fixation on DSI as the dependent variable, and diagnosis, age, and diagnosis x age as predictor variables.
Examination of genetic underpinnings of visual social attention
Intraclass correlations were used to determine the concordance among social visual attention patterns between siblings, twin pairs and a random pairing of 850 unrelated subjects.
Ethical approval statement
This study was approved by the Institutional Review Board at the University of California, San Diego (IRB #181,652, #172,066, #081,722, #041,715, #140,673, #130,352, #110,049, #070,229) and performed in accordance with the UCSD Human Research Protections Program guidelines. Prior to data collection, informed consent was obtained from all subjects and/or their legal guardians for study participation. GeoPref Test social images are copyrighted by Gaiam Americas Inc. and were used with permission.
Results
Visual attention preference stratified by diagnosis and correlation with clinical measures
Similar to our previous study25, results from the independent sample of 1,419 toddlers indicated significant differences in the amount of time a toddler fixated on DGI based on diagnostic group membership (F(6,1,412) = 43.74, p < 0.0001). There were no differences in terms of data distribution (Kolmogorov–Smirnov Z Test, D = 0.042, p = n.s.) or effect sizes between the new independent sample and our prior smaller sample of 44425 toddlers, and thus both were combined to increase power and provide the largest dataset possible for subsequent analyses (N = 1,863). See Supplemental Results and eFigure 3 for the independent sample data. Within the combined sample of 1,863, toddlers with ASD exhibited the highest percent fixation to DGI compared to all other toddler types (ASD 95% CI [40.67, 44.30] vs. ASD-Feat 95% CI [26.88, 35.03], mean difference: 11.53 ± 4.03, p < 0.0001, d = 0.47 95% CI [0.26, 0.68]; ASD vs. GDD 95% CI [24.86, 32.35], mean difference: 13.88 ± 3.46, p < 0.0001, d = 0.57 95% CI [0.38, 0.76]; ASD vs. LD 95% CI [20.30, 25.19], mean difference: 19.74 ± 7.44, p < 0.0001, d = 0.84 95% CI [0.68, 1.00]; ASD vs. Other 95% CI [21.70, 27.51], mean difference: 17.88 ± 6.15, p < 0.0001, d = 0.75 95% CI [0.57, 0.92]; ASD vs. TD 95% CI [22.03, 25.12], mean difference: 18.91 ± 7.54, p < 0.0001, d = 0.85 95% CI [0.73, 0.97]; ASD vs. TypSibASD 95% CI [17.50, 25.04], mean difference: 21.21 ± 10.29, p < 0.0001, d = 0.87 95% CI [0.61, 1.14]). Preference for DGI was also high for the ASD-Feat group (ASD-Feat vs. TD mean difference: 7.38 ± 3.51, p < 0.05, d = 0.41 95% CI [0.20, 0.62]; ASD-Feat vs. LD mean difference: 8.21 ± 3.41, p < 0.05, d = 0.44 95% CI [0.20, 0.68]). TD, LD, GDD, TypSibASD, and toddlers categorized as Other exhibited a stronger preference for DSI, and preference strength was comparable between groups. See Fig. 1. There were no differences in percent fixation levels when data was stratified by sex, although small differences were associated with ethnicity and race. See Supplemental Results eFigure 4– 6.
Since diagnosis and age were significant predictors of DGI fixation (overall R2 = 0.21, F(7,1855) = 72.27, p < 0.0001), correlation coefficients were computed for each diagnostic group separately to determine relationships between DGI fixation and clinical symptoms. For toddlers with ASD, percent DGI fixation was significantly correlated with all clinical measures and all associated subscales. In contrast, apart from the visuospatial subscale on the Mullen, there were no significant relationships found between DGI fixation and clinical profiles within the TD or TypSibASD toddlers. See Supplemental Results eTable 1.
Among the 3 ASD subtypes (see Methods), significant differences in symptom severity (ADOS total score F(2,707) = 37.21, p < 0.0001), cognitive ability (Mullen Early Learning Composite F(2,692) = 21.04, p < 0.0001), and adaptive behavior (Vineland Adaptive Behavior Composite (F(2,707) = 20.84, p < 0.0001) were observed, suggesting the possibility of unique underlying biological profiles. The largest differences were between toddlers who strongly preferred geometric (ASDGeo) and those that strongly preferred social (ASDSocial) images. See Fig. 2 a.
Figure 2.
Differences in clinical and saccade profiles across ASD subgroups. ( a) Bar graphs (mean ± SD) demonstrating clinical scores obtained from ASD toddlers with varying levels of preference for dynamic geometric images (DGI). ASD toddlers with a strong preference for DGI (≥ 69% fixation to DGI) were designated as ASDGeo. ASD toddlers with a strong preference for dynamic social images (DSI) (≥ 69% fixation to DSI) were designated as ASDSoc. Toddlers who lacked a strong preference for either stimulus were designated as ASDMid toddlers. ( b) Scatterplot showing average saccades/sec across diagnostic groups (mean ± SEM) to dynamic geometric images (DGI, y-axis) or dynamic social images (DSI, x-axis). ( c) Scatterplot showing saccades/sec to DGI (y-axis) or DSI (x-axis) for all ASD subgroups. Density plots demonstrate distribution of individual saccade patterns to DGI or DSI. ( d, e) Cohen’s d effect sizes associated with pairwise comparisons of average saccades/sec to DGI or DSI. ASD: Autism Spectrum Disorders, ASD-Feat: ASD Features, GDD: Global Developmental Delay, LD: Language Delay, TD: Typically Developing; TypSibASD: Typical Sibling of subject with ASD.
Classification accuracy: traditional approaches
ROC curves leveraging the full dataset yielded 98% specificity, 17% sensitivity, 81% PPV, and 65% NPV when the 69% fixation threshold was used. See Supplemental Results eTable 2. These values are nearly identical to validation statistics computed in previous work using the same fixation threshold16. These results indicate that even among a large population of toddlers of various developmental types, the GeoPref Test accurately distinguishes ASD from non-ASD toddlers, with relatively few false positives (i.e., 2%), which is a key criterion for biomarker tests76.
Age-binned data show that while specificity remains high throughout development (> 90%), sensitivity is consistently low. Negative predictive value is highest at 12 months and lowest by 32 months, while peak positive predictive value is achieved by 24 months. See Fig. 1 d.
We additionally examined classification statistics after stratification using demographic factors and found that the GeoPref Test performs similarly across sex, ethnic and racial groups. See Supplemental Results eTable 3.
Saccade profiles within ASD
In our previous work, we demonstrated that ASDGeo toddlers exhibited significantly fewer saccades/second when viewing DGI, but greater saccade rates when viewing non-preferred social images, in contrast to ASDSoc toddlers who had near-typical saccade patterns16. This was replicated in the current large sample when saccades/second was examined within DGI (F(8,1847) = 9.65, p < 0.0001) or DSI (F(8,1847) = 33.24, p < 0.0001) images. All planned comparisons between ASDGeo and other diagnostic groups were significant, with the largest effect sizes found between ASDGeo and TD toddlers. See Fig. 2 b-e.
Classification accuracy: cross validation with independent hold out set
To enhance the rigor of the GeoPref Test, we next performed tenfold cross validation to determine appropriate DGI fixation thresholds for computing validation statistics. Using this method, the ideal fixation threshold was 61.3%, which yields 95% specificity, 23% sensitivity, 76% PPV, 66% NPV, and 67% accuracy. When this same threshold was applied to an independent, hold-out test set, the GeoPref Test had 96% specificity, 33% sensitivity, 84% PPV, 69% NPV, and 71% accuracy.
Despite high specificity using the 69% fixation threshold16, 25 or the 61% fixation threshold obtained from tenfold cross validation, sensitivity remained low. However, combining eye-tracking measures, including saccades per second within DSI (optimal threshold from tenfold cross validation was 2.29 saccades/sec), and percent DGI fixation (61.3% fixation threshold), enhanced GeoPref Test sensitivity to 33.3%, with little impact on specificity (95.2%), PPV (81.4%) or NPV (71.2%) for the independent, hold-out test set. See Fig. 1 e.
Test–retest reliability
A key component of biomarker validation research is characterizing the stability of test performance across repeated measurements. Five hundred and thirty-five toddlers participated in repeat eye tracking. High levels of reliability were observed for more immediate, 0–1 month retests (intraclass correlation coefficient = 0.76, p < 0.0001; paired samples t-test t(75.32) = -0.72, p > 0.05). Longer interval retests were still reliable, although correlation strength was reduced. This result is expected as the GeoPref Test was created for assessing ASD during the toddler period, during which age-related visual attention changes are expected. See Supplemental Results eTable 4.
Developmental trajectory of visual social attention preference across the first years of life
Examination of the correlation between social and non-social fixation levels and age across all diagnostic groups and ASD subtypes revealed an interesting trend: both ASD toddlers with the social profile (ASDSoc) and other non-ASD toddlers significantly decreased their interest in social images with age, alongside a concomitant increase in interest in non-social images. In contrast, toddlers with the geometric profile (ASDGeo) as well as those that fell into the middle category (ASDMid) showed no age-related changes. See Fig. 3. Follow-up linear regression analyses and pairwise comparisons of beta coefficients confirm this result (overall fit: F(17,1845) = 151.5, R2 = 0.58; ASDGeo vs. TD p < 0.001, ASDGeo vs. TypSibASD p < 0.05, ASDGeo vs. GDD p < 0.0001, ASDGeo vs. LD p < 0.001, ASDGeo vs. Other p < 0.01, ASDGeo vs. ASD-Feat p < 0.001).
Figure 3.
Developmental trajectories of social and non-social (geometric) attention distinguish ASD toddlers from toddlers of other developmental types. Best fit lines depicting developmental trajectories of percent fixation to social images (blue) or non-social images (red) for toddlers of different developmental types and associated Pearson’s r and p-values. ASD toddlers were again split into the three ASD subgroups based on visual attention preference for social (ASDSoc) or non-social stimuli (ASDGeo). ASDMid toddlers did not exhibit a strong preference for social or non-social stimuli. ASD: Autism Spectrum Disorders, ASD-Feat: ASD Features, GDD: Global Developmental Delay, LD: Language Delay, TD: Typically Developing, TypSibASD: Typical Sibling of subject with ASD.
Sibling and twin correlations highlight the genetic underpinnings of visual social attention in ASD
The current sample size allows us to compare DGI fixation among a larger group of siblings than our previous work16. ASD Concordant pairs of siblings or twins exhibited the highest ICC values compared to ASD NonConcordant pairs, NonASD Concordant pairs, NonASD NonConcordant pairs, and randomly paired subjects (Fig. 4). These findings highlight the sensitivity of the GeoPref Test for detecting a genetically driven subtype of ASD.
(Truncated: full text at source URL.)