Evaluating the Feasibility of The NIH Toolbox Cognition Battery in Autistic Children and Adolescents
(Full-text capture 2026-09-21; web artifacts lightly stripped; truncated.)
Desiree R Jones
1University of Texas at Dallas, School of Behavioral and Brain Sciences, 800 W Campbell Rd, Richardson, Texas, 75080, USA
1,*, Aaron Dallman
Aaron Dallman
2University of North Carolina at Chapel Hill, 321 S. Columbia St, Chapel Hill, NC, 27514, USA
2, Clare Harrop
Clare Harrop
2University of North Carolina at Chapel Hill, 321 S. Columbia St, Chapel Hill, NC, 27514, USA
Allison Whitten
3Vanderbilt University, 1215 21st Ave, Medical Center East, Nashville, TN, 37232, USA
Jill Pritchett
4Ohio State University, 1581 Dodd Drive, Columbus, Ohio, 43210, USA
Luc Lecavalier
4Ohio State University, 1581 Dodd Drive, Columbus, Ohio, 43210, USA
James W Bodfish
3Vanderbilt University, 1215 21st Ave, Medical Center East, Nashville, TN, 37232, USA
3, Brian A Boyd
Brian A Boyd
5University of Kansas, 444 Minnesota Ave., Suite 300, Kansas City, KS, 66101, USA
5
- Author information
- Article notes
- Copyright and License information
1University of Texas at Dallas, School of Behavioral and Brain Sciences, 800 W Campbell Rd, Richardson, Texas, 75080, USA
2University of North Carolina at Chapel Hill, 321 S. Columbia St, Chapel Hill, NC, 27514, USA
3Vanderbilt University, 1215 21st Ave, Medical Center East, Nashville, TN, 37232, USA
4Ohio State University, 1581 Dodd Drive, Columbus, Ohio, 43210, USA
5University of Kansas, 444 Minnesota Ave., Suite 300, Kansas City, KS, 66101, USA
Desiree R. Jones and Aaron Dallman have contributed equally
*
Corresponding Author: desi.jones@utdallas.edu
Issue date 2022 Feb.
Terms of use and reuse: academic research for non-commercial purposes, see here for full terms. https://www.springer.com/aam-terms-v1
PMCID: PMC9817474 NIHMSID: NIHMS1700284 PMID: 33761062
The publisher’s version of this article is available at J Autism Dev Disord
Abstract
This study evaluates the feasibility of the NIH Toolbox Cognition Battery (NIH-TCB) for use in autism spectrum disorder (ASD). 116 autistic children and adolescents and 80 typically developing (TD) controls, ages 3–17 years, completed four NIH-TCB tasks related to inhibitory control, cognitive flexibility, processing speed, and episodic memory. While the majority of autistic and TD children completed all four tasks, autistic children experienced greater difficulties with task completion. Across autistic and TD children, performance on NIH-TCB tasks was highly dependent on IQ, but significant performance differences related to ASD diagnosis were found for two of four tasks. These findings highlight the potential strengths and limitations of the NIH-TCB for use in autistic children.
Keywords: Assessment, Inhibitory Control, Cognitive Flexibility, Processing Speed, Episodic Memory, Outcome Measures
The cognitive profile of autism spectrum disorder (ASD) has proven notoriously difficult to characterize ( Toplak et al., 2009). While hallmark features of ASD, such as social impairment, have been well-documented, there is little consensus on how other cognitive domains, including memory, executive function (EF), and processing speed, are impacted in ASD. Autistic children show high variability in cognitive performance, even across measures designed to assess the same construct ( Christ et al., 2011; Mohamed et al., 2018), and behavioral assessments often fail to reflect the real-world abilities and needs of these children ( Leung & Zakzanis, 2014). These findings highlight an important issue in the field of autism research: the lack of a valid, accessible measure of cognitive abilities in autistic children.
Limitations of Current Cognitive Measures
A number of factors limit the utility of common cognitive measures for use with autistic individuals. A cognitive measure that can be used across age ranges and cognitive levels is one such limiting factor. For example, widely used measures such as the Delis-Kaplan Executive Function System (D-KEFS; Delis et al., 2001) and the Wisconsin Card Sorting Task ( Grant & Berg, 1981), which assess EF and attention abilities, are restricted to school-aged children, and few assessments are available for use across autistic children and adolescents. A tool to assess trajectories of cognitive abilities throughout childhood and into adulthood is of high importance, as the severity and profile of impairments may change as children reach maturity. While inflexible thought patterns and difficulty inhibiting impulses are more prevalent in younger autistic children ( Van Den Bergh et al., 2014), autistic adolescents experience more difficulty staying organized, starting new tasks, and planning ( Rosenthal et al., 2013; Van Den Bergh et al., 2014), and show greater deficits in processing speed relative to their peers ( Iarocci & Armstrong, 2014). Therefore, it is essential that cognitive assessments for ASD are sensitive to the differential impact of age on cognitive abilities.
In addition to limitations related to child age, a variety of testing difficulties may also contribute to challenges in characterizing the full cognitive profile of autistic children. In some cases, distractibility or low engagement make it difficult for autistic children to attend to cognitive tasks for a sustained time ( Ozonoff et al., 2005). Other children may have language deficits or intellectual disabilities (ID) that contribute to poor understanding of task demands – within autistic children, approximately 33% have co-occurring ID ( Baio et al., 2018) and 30% do not acquire spoken language or are minimally-verbal ( Tager-Flusberg & Kasari, 2013). As such, the majority of existing studies of cognitive functioning in ASD have implemented IQ cutoffs and recruited individuals with fewer functional or communicative impairments. As a result of this, little is known about the cognitive profile of autistic individuals with co-occurring ID ( Stedman et al., 2019), or the validity of cognitive measures within this population. The development of a feasible cognitive assessment for use in autistic children across the age and functioning spectrum is of high importance, and has recently been highlighted as a research priority ( Demetriou et al., 2018).
The NIH Toolbox Cognition Battery
The NIH Toolbox Cognition Battery (NIH-TCB) is a set of cognitive tests designed to provide a standardized measure of cognitive abilities in clinical populations while minimizing floor and ceiling effects ( www.nihtoolbox.org; Gershon et al., 2013; Weintraub et al., 2013). Initially validated in a set of 476 healthy individuals, aged 3–85, this measure is broadly applicable across the lifespan ( Weintraub et al., 2014). NIH-TCB tasks encompass a broad range of cognitive functions, including autism-relevant constructs such as cognitive flexibility, inhibitory control, episodic memory, and processing speed. Assessments are delivered via an iPad, with animated game-like exercises that progressively increase in difficulty. Task instructions are given interactively in a simple and straightforward fashion, with practice trials included to provide further clarification of each task. The engaging format and ease of use for the NIH-TCB make this assessment especially promising for children and individuals with cognitive impairment.
Although data evaluating the performance of the NIH-TCB within individuals with intellectual and developmental disabilities is limited, preliminary evidence suggests that it may show promise within these populations. In a study encompassing individuals with Fragile X syndrome, Down Syndrome, and ID, Hessl and colleagues (2016) found that 66 to 92 percent of participants were able to complete each NIH-TCB task with valid results. A follow-up study found that among participants with ID, the majority of individuals with a mental age greater than six years could complete all tasks ( Shields et al., 2020). Furthermore, autistic adolescents and young adults in a recent study demonstrated significantly lower scores on NIH-TCB tasks assessing inhibitory control, cognitive flexibility, processing speed, and episodic memory relative to IQ-matched controls ( Solomon et al., 2020). However, no studies to date have specifically examined the performance of this measure for assessing cognitive abilities in autistic individuals across a range of ages and IQs.
The goal of the present study is to evaluate the feasibility of the NIH-TCB in a large, well-characterized sample of autistic children and adolescents, including those with IQs below the average range. Specifically, feasibility of each task was assessed in terms of a) the percentage of autistic children who could complete the task, b) the task’s performance across a range of ages and IQs, and c) the task’s ability to replicate previously observed cognitive differences between autistic and TD children. We administered four NIH-TCB tasks: 1) Flanker Inhibitory Control and Attention, 2) Dimensional Change Card Sort, 3) Pattern Comparison Processing Speed, and 4) Picture Sequence Memory Task, which were selected to encompass a range of cognitive abilities commonly studied in ASD.
We hypothesized that autistic children would complete the NIH-TCB at lower rates than TD children, and that within the ASD group, autistic children with IQs below the average range (≤ 70) would complete the tasks at lower rates than those with IQs in the average or above average range. When comparing task performance across autistic children and TD controls, we hypothesized that after controlling for child age and IQ, autistic children would score significantly lower on assessments related to inhibitory control, psychomotor processing speed, and episodic memory relative to controls, replicating previous findings ( Christ et al., 2007; Gaigg et al., 2014; Kenworthy et al., 2013; Leung & Zakzanis, 2014; Solomon et al., 2020).
Method
Participants
Autistic children were recruited from the research registries at three metropolitan universities in the southeastern and midwestern United States as part of a wider multi-site study (to be inserted following peer review); inclusion in these registries was based on expert clinical assessment. Study inclusion criteria were a) child age between 3–17 years, b) a formal diagnosis of ASD, and c) at least one parent or caregiver with English language proficiency. Exclusion criteria were limited to the presence of a comorbid genetic disorder, including Fragile X. Children in the ASD group were administered the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al., 2012) by trained and reliable assessors to confirm ASD diagnosis, and participants not meeting full diagnostic criteria (n=1) were excluded.
TD children were recruited through research registries at the same universities. Inclusion criteria included a) child age between 3–17 years, and b) at least one parent or caregiver with English language proficiency. Exclusion criteria included a) a first-degree relative diagnosed with ASD and b) diagnosis of an intellectual or developmental disability.
The final sample consisted of 196 children (116 ASD and 80 TD), aged 3–17 (Mage = 9.26, 77% male). Child and caregiver demographic information for these participants was collected by parent report. Child and caregiver characteristics are reported in Table 1.
Table 1.
Child and Caregiver Characteristics
| ASD (N =116) | TH (N = 80) | χ2 | p-value | |
|---|---|---|---|---|
| * * * | ||||
| :— | ||||
| Child Gender (%) | ||||
| * * * | ||||
| Male | 80 | 73 | 1.58 | .229 |
| Female | 20 | 27 | ||
| Child Race (%) | ||||
| * * * | ||||
| White | 73 | 76 | 3.82 | .632 |
| Black | 12 | 10 | ||
| Asian | 3 | 4 | ||
| Middle Eastern | 0 | 1 | ||
| Multiracial | 11 | 8 | ||
| Other | 1 | 1 | ||
| Child Ethnicity (%) | ||||
| * * * | ||||
| Non-Hispanic | 93 | 90 | 0.61 | .597 |
| Hispanic | 7 | 10 | ||
| Child Information [M (SD)] | ||||
| * * * | ||||
| Child Age | 9.43 (3.84) | 9.01 (3.31) | .429 | |
| ADOS-2 Overall Score | 13.49 (4.98) | NA | NA | |
| ADOS-2 Calibrated Severity Score | 7.02 (1.97) | NA | NA | |
| SB-5 Non-verbal Fluid Reasoning | 9.23 (3.85) | 11.23 (2.94) | < .001 | |
| SB-5 Verbal Knowledge | 7.65 (4.01) | 11.48 (2.08) | < .001 | |
| SB-5 Standard Score | 90.50 (21.31) | 108.44 (11.57) | < .001 | |
| SCQ Raw Score | NA | 3.41 (2.43) | NA | |
| Caregiver Gender (%) | ||||
| * * * | ||||
| Male | 6 | 10 | 1.05 | .413 |
| Female | 94 | 90 | ||
| Caregiver Race (%) | ||||
| * * * | ||||
| White | 76 | 79 | 4.35 | .513 |
| Black | 15 | 9 | ||
| Asian | 4 | 5 | ||
| Middle Eastern | 0 | 1 | ||
| Bi/Multi-racial | 5 | 5 | ||
| Other | 0 | 1 | ||
| Caregiver Ethnicity (%) | ||||
| * * * | ||||
| Non-Hispanic | 95 | 93 | 0.45 | .553 |
| Hispanic | 5 | 7 | ||
| Caregiver Education (%) | ||||
| * * * | ||||
| Some High School | 1 | 0 | 9.14 | .041 |
| High School Graduate or GED | 5 | 1 | ||
| Some College or Post-High School Degree | 29 | 16 | ||
| College Graduate | 34 | 36 | ||
| Advanced Graduate or Professional Degree | 31 | 47 | ||
| Household Income (%) | ||||
| * * * | ||||
| Less than $20,000 | 11 | 1 | 14.33 | .005 |
| 40,000 | 11 | 4 | ||
| 60,000 | 14 | 18 | ||
| 90,000 | 22 | 16 | ||
| More than $90,000 | 42 | 61 |
ASD = Autism Spectrum Disorder; TD = Typically Developing; ADOS-2 = Autism Diagnostic Observation Schedule, Second Edition; SB-5 = Stanford Binet Intelligence Scales, Fifth Edition
Measures
Autism Diagnostic Observation Schedule, Second Edition (ADOS-2)
Autistic participants were administered the ADOS-2 ( Lord et al., 2012) by trained, reliable assessors to confirm diagnosis. The ADOS-2 is a play-based, semi-structured assessment used to diagnose ASD in children and adolescents. This measure demonstrates strong predictive validity, correlating strongly with clinical diagnoses of ASD ( Gotham et al., 2007; Hus & Lord, 2014). Children were administered one of four ADOS-2 modules based on age and language ability. The majority of children in the study (N = 75) completed module three, with the remaining participants completing module one (N = 13), module two (N = 11), and module four (N= 9). To compare the severity of ASD symptoms across ADOS-2 modules, ADOS-2 calibrated severity scores (CSS; Gotham, Pickles, & Lord, 2009) were used. This scoring provides a standardized metric of ASD severity, with possible CSS values ranging from 0 to 10 and scores greater than 3 indicating a diagnosis of ASD.
Stanford-Binet Intelligence Test, Fifth Edition (SB-5)
All participants were administered the abbreviated SB-5 ( Roid, 2003) to determine verbal, nonverbal, and general IQ ( Table 2). Participants completed the Vocabulary subtest, which assesses verbal knowledge, and the Object Series/Matrices subtest, which assesses nonverbal fluid reasoning. Raw scores for verbal knowledge and nonverbal fluid reasoning were summed and standardized to generate general IQ scores. The abbreviated SB-5 battery shows strong clinical utility in autistic children, correlating highly with full-scale IQ scores ( Twomey et al., 2018). Within our sample, a total of nineteen children (9 ASD, 10 TD) lacked valid SB-5 scores. Children with missing IQ scores were significantly younger than those who completed the SB-5 (MMissing = 7.28 years, MCompleted = 10.00 years; p = .002). The majority of missing cases were due to an inability to complete the task due to behavioral limitations or lack of understanding, with one TD score missing due to invalid administration. These participants were excluded from any IQ-based analyses.
Table 2.
NIH-TCB Test Completion Rates by Diagnosis and IQ Classification
| ASD | TD | |
|---|---|---|
| * * * | ||
| :— | ||
| Below Average (N =23) | Average (N = 84) | |
| :— | :-: | :-: |
| * * * | ||
| :— | ||
| Flanker | 83% | 96% |
| DCCS | 61% | 94% |
| PCPS | 48% | 76% |
| PSMT | 48% | 69% |
ASD = Autism Spectrum Disorder; TD = Typically Developing; Flanker = Flanker Inhibitory Control and Attention; DCCS = Dimensional Change Card Sort; PSMT = Picture Sequence Memory Task; PCPS = Pattern Comparison Processing Speed
Social Communication Questionnaire (SCQ)
The Social Communication Questionnaire (SCQ; Rutter, Bailey, & Lord, 2003) is a 40item parent-report measure used to identify possible cases of ASD in children. Ratings are based on caregiver responses indicating the presence or absence (‘yes’ = 1, ‘no’ = 0) of abnormal behavior, with possible scores ranging from 0–40 and scores of 15 or higher suggesting possible cases of ASD. Previous studies demonstrate that the SCQ strongly discriminates between ASD and non-ASD cases, with a sensitivity of .88 and specificity of .72 ( Chandler et al., 2007).The SCQ was added to screen for ASD symptomatology amongst TD subjects after the study began; therefore, SCQ data were available for only 76% of children. None of these TD subjects screened positive for ASD on the SCQ (M = 3.41, SD = 2.43, range: 0–11).
NIH Toolbox Cognition Battery (NIH-TCB)
The NIH-TCB is a set of standardized, clinically validated iPad-based assessments ( Gershon et al., 2013). Normative scores for assessments are based on a nationally representative sample, with separate normative scores available for each age from 3–17 years. These age-adjusted standard scores have a mean of 100 and standard deviation of 15. As the NIH-TCB app does not allow for randomization of tasks, four tasks were chosen and administered in a fixed order: 1) Flanker Inhibitory Control and Attention, 2) Dimensional Change Card Sort, 3) Pattern Comparison Processing Speed, and 4) Picture Sequence Memory Task. All testing was completed on the same day and assessments were administered sequentially whenever possible, but children were given breaks as needed. An NIH-TCB task was discontinued and scores were considered invalid and coded as incomplete if the child failed to pass the practice items, showed evidence of poor understanding, refused to respond, required extensive prompting, or could not comply with the task demands after being given a break. In these cases, the administrator then attempted to administer the next task following the same procedure. The majority of incomplete data were due to a failure to pass the practice rounds (Flanker: n = 7, DCCS: n = 18, PSMT: n = 45, PCPPS: n = 44), with only three children discontinuing a task after passing these items (DCCS: n = 1, PCPS: n = 2).
Flanker Inhibitory Control and Attention (Flanker) is designed to assess the inhibitory and attentional aspects of EF. For this task, children were shown an arrow flanked by two additional arrows on each side and instructed to choose an on-screen button matching the direction of the middle arrow ( Figure 1a). Twenty trials were presented, with the flanking arrows alternating randomly between congruence and incongruence with the middle arrow. For children ages 3–7, this task began with a learning trial using 20 cartoon fish, with the arrow trials only appearing if at least 90% accuracy was obtained on these trials.
Fig. 1.
Still images taken from the NIH Toolbox Cognition Battery Application, depicting a) Flanker nhibitory Control and Attention, b) Dimensional Change Card Sort, c) Pattern Comparison Processing Speed, and d) Picture Sequence Memory Task. Depicted tasks are indicated for children aged 8–17. Images used with permission of NIH Toolbox.
Dimensional Change Card Sort (DCCS) is designed to assess cognitive flexibility. For this task, children were shown an on-screen target image and instructed to select one of two images matching the target on either shape or color ( Figure 1b). Participants matched images by each dimension independently for 10 trials (5 trials of shape matching and 5 of color matching), followed by a set of 30 “switch” trials, in which the task switched between matching by shape and by color.
Pattern Comparison Processing Speed (PCPS) is designed to assess psychomotor speed of processing. For this task, two simple images were presented side-by-side on screen, and children were instructed to select whether the images were the same or not ( Figure 1c). Images were presented in succession over a period of 85 seconds, with the participant responding to as many images as possible within this period. This task is motorically demanding and is thus a measure of psychomotor processing speed, as opposed to informational processing speed.
Picture Sequence Memory Test (PSMT) is designed to assess episodic memory abilities. For this task, participants were shown a narrated sequence of images corresponding to common activities; for instance, participants were shown the theme of “How to go to the Park”, followed by images depicting children playing on swings, feeding ducks, etc. ( Figure 1d). Participants were then asked to recall the order of the presented images by dragging them in the correct order on screen. The same sequence was repeated for a second trial, with additional images added and the participant was again asked to recall the proper sequence of events. Sequences varied from 6 to 18 pictures, based on the participant’s chronological age.
Data Analysis Plan
Age-corrected standard scores were computed within the NIH Toolbox Application based on the participant’s raw score relative to the NIH Toolbox normative data for their age. Raw scores for the Flanker and DCCS tasks were generated by combining response time and accuracy vectors. For participants with less than 80% accuracy across trials, only accuracy was used to generate performance scores. Raw scores for PCPS age-corrected standard scores were based on the number of items answered correctly in the 85-second period. For PSMT, raw scores were generated based on the number of adjacent image pairs placed in the correct order for each trial. Age-corrected standard scores were chosen in order to account for the broad age ranges evaluated in this study. All analyses were completed using SPSS 23.0 ( IBM SPSS Inc., 2015).
The percentage of children completing each task, as well as the number of total tasks completed by children, were used to assess the feasibility of the NIH-TCB tasks in autistic children. The proportion of children completing each individual task, and the proportion of children who completed all four tasks, were calculated for children based on diagnostic group and IQ. For the purpose of this analysis, children were divided among three IQ categories: “Below Average” (≤ 70), “Average” (71–130), and “Above Average” (131+), with data reported separately for children with missing IQ scores (N = 13). The majority of children fell within the “Average IQ” range (N = 152). Chi-square tests of independence with exact estimation were used to compare these completion proportions across diagnostic groups. Students’ t-tests were used to compare the total number of NIH-TCB tasks completed for autistic children compared to TD children. Due to small and unequal sample sizes, no direct comparisons were made between IQ groups.
To determine the degree to which IQ and ASD diagnosis contribute to age-corrected standard scores for each NIH-TCB task, hierarchical multiple regressions were run with three blocks: block 1 included child age, block 2 included IQ, and block 3 included diagnostic group. Child age and IQ were treated as continuous predictors and grand mean centered, while diagnostic group was dummy coded and used as a categorical predictor, with ASD diagnosis as the reference group. To encourage parsimony and avoid overfitting of the model ( Vandekerckhove et al., 2014), predictors that did not significantly improve model fit were dropped from the final regression model.
Results
Task Completion Metrics
Table 2 shows task completion rates for each NIH-TCB task grouped by diagnostic group and IQ classification. Overall, 96% of autistic children and 99% of TD children completed at least one of the four tasks. Completion rates were highest for the Flanker task and did not differ significantly between diagnostic groups (χ2 = 1.32, p = .37), with 92% of autistic children and 96% of TD children completing this task. The proportion of children completing each assessment was significantly higher for the TD group across the DCCS (ASD = 82%, TD = 95%; χ2 = 7.31, p = .008), PCPS (ASD = 67%, TD = 89%; χ2 = 12.02, p = .001), and PSMT (ASD = 64%; TD = 90%; χ2 = 17.11, p < .001) tasks. The proportion of children completing all four tasks also differed significantly as a function of diagnosis, with 88% of TD children completing all four NIH-TCB tasks, compared to 57% of autistic children (χ2 = 20.88, p < .001). On average, autistic children completed significantly fewer NIH-TCB tasks compared to TD children (MASD = 3.05, SD = 1.24, MTD = 3.70, SD = 0.83; t(194) = 4.387, p < .001).
Across autistic children and TD children, the percentage of children completing the tasks varied widely among IQ groups. In terms of overall task completion, 100% of children with above average IQ and 79% of children with average IQ completed all four assessments, compared to only 35% of children with below average IQ and 37% of children with missing IQ scores. Among children with below average IQs (N = 23), completion rates were highest for the Flanker task (83%) and lowest for the PCPS and PSMT tasks (48%). These children completed an average of 2.39 tasks out of four, with 87% completing at least one task and 35% completing all four tasks. Children with missing IQ scores (N = 19) performed similarly to children with below average IQ, with an average of 2.42 tasks completed. Ninety percent of children with missing IQ scores completed at least one task, and 37% of these children completed all four tasks.
Figure 2 depicts the proportion of participants in each IQ class who completed from zero to four of the tasks. Although children with average IQ completed the NIH-TCB assessments at a higher rate than those with below average or missing IQ scores, diagnostic differences in task completion were apparent. Among children with IQs in the average range, autistic children (N = 84) completed significantly fewer tasks than TD children ( M ASD = 3.36, SD = 1.01, MTD = 3.80, SD = 0.64; t(142) = 3.30, p = .001).
Fig. 2.
Percentage of participants completing each proportion of tasks. Percentages represent the proportion of participants within each IQ classification who completed from zero to four tasks. For IQ classifications, below average = IQ ≤ 70, average = IQ 71–130, above average = IQ ≥ 131.
Impact of Age, IQ, and Diagnosis on Standard Scores
Means, standard deviations, and regression coefficients for the final block of each regression model, with all non-significant predictors omitted, are reported in Table 3. Child age alone contributed significantly to NIH-TCB age-corrected standard scores for the Flanker ( F(1, 168) = 7.73, MSE = 268.27, p = .006), PCPS ( F(1,137) = 7.07, MSE = 483.95, p = .009), and PSMT tasks ( F(1, 131) = 12.90, MSE = 304.70, p < .001), but not the DCCS task ( F(1, 160) = 1.75 , MSE = 276.05, p = .188). The addition of child IQ significantly increased the proportion of explained variance in NIH-TCB scores for all four measures: Flanker: ( F(2, 167) = 23.32 , MSE = 220.66, p < .001), DCCS: ( F(2,159) = 32.26, MSE = 199.77, p < .001), PCPS: ( F(2,136) = 14.72, MSE = 421.43, p < .001), and PSMT: ( F(2, 130) = 13.95, MSE = 277.67, p < .001). When diagnostic group was added to the model, it contributed to small, but significant improvements in model fit for the Flanker ( F(3,166) = 18.15, MSE = 213.86, p < .001) and PCPS tasks ( F(3,135) = 12.87, MSE = 401.64, p < .001), but not the DCCS ( F(3,158) = 22.75, MSE = 197.35, p < .001) or PSMT tasks ( F(3,129) = 9.24, MSE = 279.78, p < .001).
Table 3.
Factors contributing to variance in NIH-TCB Age-Corrected Standard Scores by Taska
| M | SD | b | β | ΔR2 | |
|---|---|---|---|---|---|
| * * * | |||||
| :— | |||||
| Flanker | 90.21 | 16.70 | 93.99 ** | − | − |
| Age | 9.26 | 3.63 | −.715 | −.156 | .044 |
| IQ | 97.59 | 20.08 | .214 ** | .338 | .174 |
| Diagnosis | − | − | −6.38 ** | −.188 | .029 |
| * * * | |||||
| DCCS | 93.62 | 16.65 | 93.62 ** | − | − |
| IQ | 97.59 | 20.08 | .444 ** | .536 | .287 |
| * * * | |||||
| PCPS | 90.34 | 22.48 | 96.66 ** | − | − |
| Age | 9.26 | 3.63 | 1.66 ** | .269 | .049 |
| IQ | 97.59 | 20.08 | .290 ** | .259 | .129 |
| Diagnosis | − | − | −10.69 ** | −.234 | .044 |
| * * * | |||||
| PSMT | 99.42 | 18.23 | 99.42 ** | − | − |
| Age | 9.26 | 3.63 | −1.32 ** | −.263 | .090 |
| IQ | 97.59 | 20.08 | .270 ** | .297 | .087 |
Flanker = Flanker Inhibitory Control and Attention; DCCS = Dimensional Change Card Sort; PSMT = Picture Sequence Memory Task; PCPS = Pattern Comparison Processing Speed
a
Predictor variables were centered around their mean prior to computing the regression. Regression coefficients reported are from the final regression, with nonsignificant predictors omitted.
*
p < .05
**
p < .01
For the Flanker task, main effects of child age, IQ, and diagnostic group accounted for approximately 25% of variance in age-corrected standard scores. When controlling for child age and IQ, autistic children performed worse than TD children ( p = .013). Approximately 22% of variance in PCPS scores was accounted for by child age, IQ, and diagnostic group. When controlling for child age and IQ, autistic children also performed worse on this task compared to TD children ( p = .006).
Discussion
Our results highlight the potential of the NIH-TCB as an integrated measure of cognitive abilities for the majority of autistic children. Across each task, at least two-thirds of autistic children successfully completed all items. While most of the TD children and autistic children within our sample were able to complete all four NIH-TCB tasks, there appear to be some limitations in who will be able to complete certain tasks. IQ contributed significantly to performance on NIH-TCB measures; autistic children with an IQ below 70 experienced difficulties completing the PCPS and PSMT tasks and completed fewer tasks on average than autistic children with an IQ above 70. Across the full sample of autistic children and TD children, child IQ contributed strongly to performance on all four tasks, with children with lower IQs scoring significantly lower compared to children with high IQs. These findings suggest that autistic children with co-occurring ID may experience difficulties understanding task instructions, contributing to failed practice items and invalid scores.
Furthermore, child age contributed significantly to scores on three of the four tasks, despite the use of age-corrected standard scores, suggesting that NIH-TCB standardized norms may not generalize to all populations. While older children tended to perform better on the PCPS task than younger children, they surprisingly performed worse on the Flanker and PSMT tasks. One possible explanation for this discrepancy is that while NIH-TCB tasks are age-appropriate and engaging for younger children, they may not adequately capture the attention of older children and adolescents, leading to decreased attention and worse performance. Further work with this population, including direct comparisons of task performance across age groups, is needed to better understand the performance of these tasks in adolescents across clinical populations.
Despite a relatively low overall completion rate in autistic children with below average IQ, an overwhelming majority of these children completed at least one task, with acceptable to high completion rates for the Flanker and DCCS tasks. These findings suggest that while the NIH-TCB Flanker and DCCS tasks demonstrate good feasibility in autistic children, further adaptations may be needed when using the PCPS and PSMT tasks on autistic individuals with co-occurring ID. In cases of ID, administration of NIH-TCB tasks based on mental age as opposed to chronological age has previously been shown to increase feasibility among participants with Down syndrome or Fragile X, although participants with a chronological or mental age below 3 years may continue to experience difficulties completing these tasks ( Hessl et al., 2016). While attrition was highest for participants with below average IQ, attrition for the last two tasks was higher across all diagnostic and IQ groups, suggesting that testing fatigue may have also contributed to a decline in motivation. Therefore, allowing additional time for breaks, providing rewards or reinforcements for task engagement, or even administering tasks over multiple days may help to maintain attention and motivation when administering multiple assessments ( Thompson et al., 2018).
Performance on the NIH-TCB Flanker and PCPS tasks in our ASD sample replicated previously observed differences that characterize the cognitive phenotype of ASD. Although autistic participants did not perform significantly worse than controls on the DCCS task, this finding is consistent with previous findings of intact cognitive flexibility in autistic children for performance-based measures ( Kenworthy et al., 2008; Leung & Zakzanis, 2014). While episodic memory of events has not been well-studied in autistic children, performance on the PSMT task was inconsistent with previous evidence of impaired episodic memory for words and pictures in autistic children ( Hala et al., 2005; Poirier et al., 2011), as well as impaired episodic event memory in autistic adults ( Lind et al., 2014). This finding may relate to how these stimuli are presented and scored; although images were presented individually, the overall sequence of images presented for the PSMT was relational, and scores were derived based on the number of image pairs correctly ordered. As previous evidence has shown that adolescents experience difficulties recognizing images presented alone, but not images presented in relation to another image ( Solomon et al., 2016), the format of this task may therefore fail to capture the specific memory difficulties found in ASD. However, due to a lack of conclusive evidence on episodic story memory in autistic children, it is difficult to draw concrete conclusions regarding the performance of this task in this population.
When controlling for age and IQ effects, significant performance differences related to ASD diagnosis were found for the Flanker and the PCPS tasks but not the DCCS or PSMT tasks. These results are consistent with previous studies of specific aspects of cognitive functioning in ASD that have reported a range of cognitive performance in autistic children, including areas of relative strength as well as areas of relative weakness. The pattern of task-specific effects that we found suggests that the NIH-TCB is capable of identifying both intact and deficient aspects of the cognitive phenotype in ASD.
Limitations
Several limitations should be considered in the context of the current study. First, no other measures of cognitive abilities were collected in addition to the NIH-TCB. Therefore, our findings reflect feasibility of use as opposed to validity, and future studies could examine the concordance of NIH-TCB and other established cognitive tasks (e.g. Wisconsin Card Sorting Task, D-KEFS). Child language ability was also omitted from the current study; future studies investigating how expressive and receptive language abilities, as opposed to general IQ, impact task performance are encouraged. Additionally, missing IQ data for several participants may have impacted the findings of this study, especially those from analyses examining the role of IQ in task performance and completion. However, missing IQ data are common in ASD studies, especially those that involve autistic individuals with co-occurring ID, and analyses were chosen to maximize power in cases of missing data. Furthermore, we found that some NIH-TCB tasks could be completed by a majority of autistic children with a below average IQ in our sample, including many who could not complete a standardised IQ test. Thus, certain NIH-TCB tasks may even provide a feasible and reasonable proxy for IQ in work with autistic children with ID, and thereby overcome a significant limitation of IQ tests in this population. An additional limitation of our study is that the non-randomized order of assessments makes it difficult to compare performance across tasks or determine the mechanisms contributing to low completion rates for the PCPS and PSMT tasks. As presently designed, the NIH-TCB batteries are administered in a set order and cannot be randomized. As completion rates were highest for the first two tasks and declined over time, the low completion rates for the PCPS and PSMT tasks may have been the result of test fatigue or declines in motivation. Thus, the proposed accommodations for maintaining attention and motivation across tasks are important considerations for future NIH-TCB administration. Finally, the present study included predominantly white, relatively high-income families, with a limited number of female participants. Future studies using the NIH-TCB on a more diverse range of participants will help to determine whether the NIH-TCB demonstrates similar metrics within a wider population of autistic children.
Conclusions
Our study provides evidence demonstrating acceptable feasibility of the NIH-TCB on a large, well characterized sample of autistic children, including a high percentage of those with an IQ below 70, a group frequently excluded from studies of cognitive function. This supports the core goal of the NIH-TCB: to allow for comparisons across a variety of studies and study populations ( Weintraub et al., 2013). Although completion rates were lower for the PCPS and PSMT tasks, many participants, both overall
(Truncated: full text at source URL.)