Quality of autism early intervention research questioned
autside.substack.com
New research published in the British Medical Journal reveals that low-quality studies dominate autism early intervention research and inform clinical recommendations in the United States. Led by Micheal Sandbank, Ph.D. of the University of North Carolina School of Medicine, the comprehensive analysis found many popular interventions lack robust evidence and monitoring for potential harms.
As we well know, the Centers for Disease Control reports rising autism rates among young children. Clinicians aim to support development through early interventions provided intensively or embedded in family routines. However, researchers disagree on optimal approaches. Sandbank’s team found the number of studies, including more accurate randomized controlled trials, has doubled in four years.
Yet even these better-designed studies fail to control key biases threatening result accuracy. Many don’t address “placebo-by-proxy” bias where caregivers overestimate improvements because they expect interventions to work. They also fail to control “detection” bias where researchers overestimate effects because they know which participants received interventions.
Considering only evidence guarding against these risks leaves little support for popular interventions. This does not mean ineffectiveness but signals need for more rigorous studies assessing benefits and harms, Sandbank says. In the meantime, clinicians should increase transparency around current evidence limitations when presenting options to caregivers struggling to support child development and their own mental health.
Wait, … what?
This updated meta-analysis highlights several concerning issues regarding the failure to control for biases in studies of autism early interventions:
Despite a doubling of randomized controlled trials in just 4 years, risks of bias remain highly prevalent. Nearly two-thirds of outcomes were subject to detection bias due to non-masked assessors, and nearly half relied on biased caregiver/teacher reports. Only a small fraction of studies properly controlled for both risks.
When considering evidence just from randomized trials, several interventions showed statistically significant benefits. However, as quality thresholds increased by excluding biased outcomes, fewer effects could be estimated with confidence and crossing significance thresholds.
For naturalistic developmental behavioral interventions (NDBIs) (a form of ABA), the only statistically significant effect remaining after excluding biased outcomes was on “core autism features.” But it's unclear if such changes are desirable outcomes for autistic individuals. Measures of autism characteristics also capture neutral or beneficial aspects.
Researchers overwhelmingly fail to adequately monitor or report potential intervention harms. No studies mentioned evaluating potential long-term negative impacts. This prevents properly weighing benefits against unintended consequences in clinical decision making.
Thus, despite research proliferation, study quality remains highly problematic in this field. Risks of bias threaten the validity of apparent intervention benefits for young autistic children. And the near complete lack of harm monitoring precludes fully informed support recommendations. Physicians should communicate these limitations transparently to families considering early interventions. More rigorous controlled trials are badly needed, as are systematic efforts to track adverse impacts, before confidence can be placed in intervention efficacy claims.
The AutSide is a reader-supported publication. To support my work, consider becoming a paid subscriber.
Quality of autism early intervention research questioned
Quality of autism early intervention research questioned
Quality of autism early intervention research questioned
New research published in the British Medical Journal reveals that low-quality studies dominate autism early intervention research and inform clinical recommendations in the United States. Led by Micheal Sandbank, Ph.D. of the University of North Carolina School of Medicine, the comprehensive analysis found many popular interventions lack robust evidence and monitoring for potential harms.
As we well know, the Centers for Disease Control reports rising autism rates among young children. Clinicians aim to support development through early interventions provided intensively or embedded in family routines. However, researchers disagree on optimal approaches. Sandbank’s team found the number of studies, including more accurate randomized controlled trials, has doubled in four years.
Yet even these better-designed studies fail to control key biases threatening result accuracy. Many don’t address “placebo-by-proxy” bias where caregivers overestimate improvements because they expect interventions to work. They also fail to control “detection” bias where researchers overestimate effects because they know which participants received interventions.
Considering only evidence guarding against these risks leaves little support for popular interventions. This does not mean ineffectiveness but signals need for more rigorous studies assessing benefits and harms, Sandbank says. In the meantime, clinicians should increase transparency around current evidence limitations when presenting options to caregivers struggling to support child development and their own mental health.
Wait, … what?
This updated meta-analysis highlights several concerning issues regarding the failure to control for biases in studies of autism early interventions:
Despite a doubling of randomized controlled trials in just 4 years, risks of bias remain highly prevalent. Nearly two-thirds of outcomes were subject to detection bias due to non-masked assessors, and nearly half relied on biased caregiver/teacher reports. Only a small fraction of studies properly controlled for both risks.
When considering evidence just from randomized trials, several interventions showed statistically significant benefits. However, as quality thresholds increased by excluding biased outcomes, fewer effects could be estimated with confidence and crossing significance thresholds.
For naturalistic developmental behavioral interventions (NDBIs) (a form of ABA), the only statistically significant effect remaining after excluding biased outcomes was on “core autism features.” But it's unclear if such changes are desirable outcomes for autistic individuals. Measures of autism characteristics also capture neutral or beneficial aspects.
Researchers overwhelmingly fail to adequately monitor or report potential intervention harms. No studies mentioned evaluating potential long-term negative impacts. This prevents properly weighing benefits against unintended consequences in clinical decision making.
Thus, despite research proliferation, study quality remains highly problematic in this field. Risks of bias threaten the validity of apparent intervention benefits for young autistic children. And the near complete lack of harm monitoring precludes fully informed support recommendations. Physicians should communicate these limitations transparently to families considering early interventions. More rigorous controlled trials are badly needed, as are systematic efforts to track adverse impacts, before confidence can be placed in intervention efficacy claims.
The AutSide is a reader-supported publication. To support my work, consider becoming a paid subscriber.