Where’s the evidence in evidence-based therapies

The long-term impacts of early childhood ABA on autistic people

Base journal article under review: Hume, K., Steinbrenner, J. R., Odom, S. L., Morin, K. L., Nowell, S. W., Tomaszewski, B., ... & Savage, M. N. (2021). Evidence-based practices for children, youth, and young adults with autism: Third generation review. Journal of Autism and Developmental Disorders, 51(11), 4013-4032.

Introduction

            The idea behind the choice of the article under review in this article comes from an interaction at a recently attended professional development session. The presenter was providing dangerous information in her talk about how to “manage” autistic student behaviours in the classroom. When I offered an alternative view, several things happened. First, the presenter engaged in “Appeal to Authority” in citing revision two of the current article as superseding the evidence I had provided, as well as superseding my lived experience of having been an autistic person subjected to the practices she was suggesting – and being left to deal with the long-term effects. Second, she later complained to District training staff about my inquiries. Third, the District staff then complained to my sponsors about my actions. Given that the session was recorded (Zoom) and that I was nothing but polite in my discussion with the presenter, nothing became of the event in terms of discipline proceedings. Interestingly, I was told by my sponsoring agency’s staff that District professional development sessions should be treated as monologues. The staff do not want, nor do they appreciate, comments or critiques other than the banal surveys that are required to be completed by all attendees for quality improvement purposes.

            When brainstorming for the week’s articles, I became aware of a third revision to the on-going project that you will read about below. Figure 1 (below) illustrates where the journal article is placed in the overall scheme of so-called evidence-based practices. The journal article seeks to ferret out and list those practices that can then be operationalised into “user-friendly procedures” and used on autistic people. Given my history, and the interactions at my school site and with my District, I made the decision to review “Evidence-based practices for children, youth, and young adults with autism: Third generation review” from volume 51 (2021) of the Journal of Autism and Developmental Disorders.

Summary of Primary Article

This systematic review journal article describes a set of practices that the authors allege provide evidence of positive effects with autistic children and youth. It represents the third iteration of a review of the intervention literature that began in 2010. The second edition of the review was conducted in 2015. This third edition extends the coverage to articles published between 1990 and 2017. Their search initially yielded 31,779 articles, and the subsequent screening and evaluation process found 567 studies to evaluate. Combined with the previous reviews, 972 articles were synthesized, from which the authors found only 28 focused intervention practices that met the criteria for a so-called evidence-based practice (EBP). [28/31770 = 0.00088]

The article begins with a summary of a medical model of disability view of autism. The authors note that “the increased prevalence of autism” has intensified the demand for “effective educational and therapeutic services,” and that “intervention science is providing mounting evidence about practices that positively impact outcomes.” The purpose of this study was to identify a set of focused intervention practices that have clear scientific evidence of positive effects with autistic children and youth (i.e., EBP). The authors do not state what it is about the autistic experience that requires such services.

This paper primarily concerned focused intervention practices and did not consider comprehensive program models. These are the two broad classes of interventions that appear in the research literature. Focused intervention practices are designed to address a single skill or goal of a person identified as deficient in that area. These practices are operationally defined and address specific learner outcomes. Teachers, clinicians, parents, or other practitioners select and use the practices (e.g., prompting, reinforcement, time delay, etc.) in interventions or instruction that addresses a learner’s individual learning goal. In the authors’ view, focused intervention practices are considered the building blocks of educational programs for autistic children and youth (see Figure 1). This is an interesting position to take given that special education covers a wider age range by statute (PK-22).

            In contrast, comprehensive program models consist of a set of practices designed to achieve a broad learning or developmental impact on core deficits. Comprehensive programs are organized around a conceptual framework, procedurally manualised, focus on a breadth of outcomes, and are implemented with a substantial number of hours per week across one or more years. Examples of such programs are the early intensive behavior intervention program based on the UCLA Young Autism Project (Smith et. al., 2000), the LEAP preschool model (Strain & Bovey, 2011), and the Early Start Denver Model (Dawson et. al., 2012). The authors note the problems that can arise when teachers or other professionals adopting comprehensive program models must commit to being trained and implementing the entire model with fidelity. Because of the differences between the two classes of interventions, and to need to specify a clearly articulated and practical focus for the review, comprehensive program models were not included in the review.

            During their comprehensive review, the authors noted several problems in the demographic data for the surveyed studies. For example, more than 50% of the participants in the surveyed studies were elementary school students; whilst middle school aged participants representing approximately 17%, high school aged participants representing approximately 10%, and less than 5% belonging to the 19-22 year old age group. In studies that reported the number of participants in the gender or sex categories, 84% of participants were male. And, although “non-binary” and “other” were included as options during the data extraction, no studies reported these categories. Only thirty percent of the examined studies reported data on race/ethnicity/nationality. For studies that reported numbers of participants by categories, 59% of the participants were White, 10% were Black, 9% were Asian, and 8% were Hispanic/Latino. All other groups had less than 5% representation among participants in studies reporting this information. As for the type of study conducted, single case design studies made up 83% of the articles. Finally, data on implementation characteristics were only extracted from later studies. Research staff implemented (i.e., directly provided) interventions in 52% of the studies and were coaches in 10% of studies. Educators and related service providers were each identified as implementers in 20% of studies, and parents were noted as implementers in 10% of studies. In 48% of studies, the intervention took place in educational settings. The other intervention locations were university clinic/research lab settings (20% of studies), home settings (18%), and community clinic settings (13%). This overwhelming problem was commented upon recently in the Lancet (Lord, et. al., 2022) citing the article presently under review, “in the USA, autism intervention research began with behavioural approaches (e.g., applied behaviour analysis) that used the manipulation of the onset, offset, and resumption of treatment approaches across single cases rather than randomised controlled trials as a way of comparing different conditions. Such research designs are systematic, inexpensive, and flexible in their ability to address the needs of different children. Yet, they have clear limitations, such as biases associated with small sample sizes, absence of information on generalisation and the role of development, and, often, non-randomisation or non-blind outcome assessments.”

Putting this together, most of the research comes from working with white male elementary school students in single-case studies conducted in controlled environments outside of educational settings. They studies suffer from small sample sizes. They lack information on how the results of each single case can be generalized to a larger population. Finally, none consider the long-term effects of the intervention. Thus, it seems hard to grasp that any of these practices would qualify as evidence-based for use in a predominantly Latino (88%), urban school district, such as the one for which I work. The verdict thus becomes that there is no evidence to be found in the journal article that most of the suggested interventions would be effective within any urban or Title I setting. The data supports no conclusion that interventions applied in mostly clinical settings to mostly young white males will work with any other population of autistic students.

An interesting omission was any consideration of the long-term efficacy or effects of the interventions. None revisited the studied population to check in and see how they were doing later in their school careers, or later in life. Unfortunately, this myopic view of test subjects is a feature in small scale action research studies such as the ones under review in the reviewed paper.

Nevertheless, the authors persisted in their analysis and list the interventions that they believe should be included in a listing of EBPs in Table 2. Of the listed interventions, all but two (Direct instruction (DI), Music-mediated intervention (MMI)) are rooted in Applied Behavioural Analysis (ABA). As we will see in the next section, their lack of awareness of the long-term effects of ABA on autistic people. Unfortunately, that lack of awareness has come at an awful price.

Commentary - Where’s the evidence in evidence-based therapies: the long-term impacts of ABA on autistic people

When you purchase an automobile, you have access to mountains of data from the manufacturer. There’s the sales sticker in the window that lets you know the important specifications. There’s information from the insurance and safety companies that let you know if you’ll likely survive a crash. There’s durability and customer service data as well that can inform you as to the reliability and expected annual costs of owning your new car. This is great. Armed with this information, you can make the right choice for you and your family.

If only we could have some similar bank of data to inform us before we have children. Sure, there’s DNA profiles and family histories. But the results of these surveys come in the form of probabilities. So many parents get their hopes set on the chance of a certain profile for their unborn child – intelligence, appearance, skills, etc. Then, several months into being a new parent and working with their new child, they suspect that something’s “wrong.” Their suspicions are confirmed at their school psychologist’s office when their child is “diagnosed” as autistic. What now?

Likely, the school’s office or IEP team will present parents / caregivers with a large packet of paperwork on autism and available “therapies” that was prepared by a non-profit autism organization (my school site currently shares this link with parents and SPED staff: https://whyy.org/articles/10-helpful-resources-for-autism-awareness-month/). So much guidance is offered to parents about the need to engage in so-called evidence-based therapies as soon as possible. Parents assume that they’re being presented with a balanced view of the available science as well as a well-rounded summary of the potential positive and negative effects. You would think that … and you would be wrong. Missing from most information packets is a fair treatment of the long-term impacts of intensive early-childhood therapy regimes (like ABA) on autistic children. 

Let’s look, shall we, at what’s missing.

According to Sandoval-Norton and Shkedy (2019), ABA is a form of behaviour modification that relies heavily on external reinforcement, both positive and negative. ABA is intended to modify or diminish behaviours, as well as increase language, communication, social skills, attention, etc. The main tenets of ABA follow behaviourist theories that suggest that behaviour is caused by external stimuli in the environment, which is why a reward (external) would reinforce a behaviour, and punishment (external) would discourage a behaviour. Whilst such conditioning may be effective for teaching specific tasks in certain situations, in nearly all other circumstances it is not typically used to the extreme extent that it has been applied with for the “treatment” of many autistic children. Take toilet training for example; many children are toilet trained using ABA’s operant conditioning techniques. However, this conditioning only applies to the one skill—and once the child has mastered it, the conditioning subsides. On the other hand, many autistic children are taught the same task or skill for years using the same conditioning techniques, yet mastery is never met. Why?

ABA therapy has been viewed as the gold standard for “treating” autistic children because various meta-analyses have found it to be very efficacious (Eldevik, et. al., 2009; NYS DHEIP, 1999; Virués-Ortega, 2010). However, research indicates efficacy only with those who have a measurable Intelligence Quotient (IQ), typically at 70 or above (Virués-Ortega, 2010). Many studies use IQ to measure efficacy or as an inclusion criterion, which means children who are nonverbal, particularly those deemed as “lower functioning” and untestable, are inherently excluded from these studies (Virués-Ortega, 2010). Therefore, nearly all research on ABA efficacy necessarily excludes most of the autistic population; yet this is the population that tends to receive continual ABA services over a longer period since they often do not meet the criteria needed for mastery of tasks for months or even years.

Ivar Lovass (1996), the pioneer of ABA, when describing criteria for proposed behavioural treatments, stated that it should only be used with auditory learners, as visual learners “do not recover with behavioural treatment.” There have been limited, if any, scientifically validated studies on the use of ABA on non-verbal autistic children or those children who experience auditory processing difficulties. Therefore, the most vulnerable children cannot be tested and as such should not be candidates for such behavioural therapies.

Why then is such intensive conditioning utilized on a population that would never be included in the studies that advocate for the usage of such treatments? Where’s the evidence in the so-called evidence-based treatments?

Moreover, despite having produced a generation of non-verbal children who have undergone many years of ABA therapy, often well into adulthood, there is limited research on the status of these individuals, most notably their propensity for independence (see Reser’s (2011) Solitary Forager Hypothesis) and whether ABA therapy has helped or hurt these individuals.

Autistic children are often exposed to operant conditioning in all aspects of their lives, until they are aged out at sometime between age 18 and 22 years when services are no longer required by law (Lewis, et. al., 2021). In fact, schools, ABA specialists, and researchers are learning that such intensive and chronic conditioning has instead amounted to compliance, low intrinsic motivation, and lack of independent functioning—the latter of which is the presumed goal of ABA therapy in the first place (Wilson, et. al., 2014). Perhaps because ABA therapy is considered effective in typical children for select tasks, the assumption is that an even more intensive approach would be suitable for non-verbal and/or “lower functioning” autistic children. Regardless, research (Sandoval-Norton & Shkedy, 2019) has indicated numerous problems with the underlying theory of ABA, specifically unintended consequences such as prompt dependency, amongst other issues (Bryan & Gast, 2000; Mesibov, et. al., 2004). Whilst conditioning and prompting is initially meant to facilitate learning and help accommodate challenges related to autonomous functioning, the usage of prompting often does not fade even when the need for it has. Prompting is used to compensate for challenges related to independent functioning and was intended as a temporary aid (Bryan & Gast, 2000; Mesibov, et. al., 2004). However, research has consistently found that individuals respond to the prompts instead of to the cues that are expected to evoke a target behaviour, ultimately contributing to learned helplessness and arguably to low self-esteem (MacDuff, et. Al., 2001; Sternberg & Williams, 2010). This may also explain why intensive conditioning in non-verbal autistic children rarely generalizes to other tasks, assuming the targeted task is even mastered.

Regrettably, the damage done by ABA therapy through this kind of intensive conditioning goes beyond adult reliance and learned helplessness. There is little evidence of prompts fading to decrease dependence and encourage students to respond to other people and more naturally occurring cues. In one study, dependence was even observed on playgrounds when a child could clearly engage in a task or play autonomously but hesitated when a paraprofessional was near (Giangreco, et. al., 1997). The proximity, constant prompting, and intensive conditioning has produced various issues that proponents of ABA therapy, and child advocates in general, have failed to study. Research (Sandoval-Norton & Shkedy, 2019) has indicated many problems with the premises behind ABA therapy and various similar interventions, yet longitudinal research examining the lives of the adults who have been subjected to such conditioning since childhood is few and far between (Wilson, et. al., 2014; Mesibov, et. al., 2004).

To conclude, when considering the lack of research on the efficacy of ABA in “treating” autistic children and the lack of research in support of long-term, intensive ABA therapy in the autistic population, as well as the lack of longitudinal research for an entire generation of autistic children who have undergone this treatment, it is evident that we need to pause and look at what has resulted. It is important to read and to understand the research (Sandoval-Norton & Shkedy, 2019) that is available, as it indicates negative effects surrounding the interventions and methods used in ABA therapy. Compliance, learned helplessness, food/reward-obsessed, magnified vulnerabilities to sexual and physical abuse, low self-esteem, decreased intrinsic motivation, robbed confidence, inhibited interpersonal skills, isolation, anxiety, suppressed autonomy, prompt dependency, adult reliance, etc., continue to be created in a marginalized population who are unable to defend themselves. ABA proponents have utilized predominantly non-verbal and neurologically different, children who are not recognized under this paradigm to have their own thought processes, basic needs, preferences, style of learning, and psychological and emotional needs, for their experiments. These children are the population that was chosen to be the subjects of an experimentally intense, lifelong treatment within a therapy where most practitioners are ignorant regarding the autistic brain—categorically, this cannot be called anything except abuse.

Follow-Up

If you’re concerned about addressing the needs of autistic students in your classroom, there is hope. Remember those two EBPs found in Figure 2 that weren’t a subset of ABA? One of those, Direct Instruction, is explored in the companion piece to this review. You can find my follow up to this entitled “Classroom Management and Alexithymia: Understanding how classroom energy states affect autistic student behaviour” in a next week’s postings.

References

Bryan, L. C., & Gast, D. L. (2000). Teaching on-task and on schedule behaviors to high- functioning children with autism via picture activity schedules. Journal of Autism and Developmental Disorders, 30, 553–567.

Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). Applied behaviour analysis (2nd ed.). Upper Saddle River, NJ: Pearson Education.

Eldevik, S., Hastings, R. P., Hughes, J. C., Jahr, E., Eikeseth, S., & Cross, S. (2009). Meta- analysis of early intensive behavioral intervention for children with autism. Journal of Clinical Child & Adolescent Psychology, 38(3), 439–450. doi:10.1080/15374410902851739

Giangreco, M. F., Edelman, S. W., Luiselli, T. E., & MacFarland, S. Z. (1997). Helping or hovering? Effects of instructional assistant proximity on students with disabilities. Exceptional Children, 64(1), 7–18.

Hume, K., Loftin, R., & Lantz, J. (2009). Increasing inde- pendence in autism spectrum disorders: A review of three focused interventions. Journal of Autism and Developmental Disorders, 39(9), 1329–1338. Individuals With Disabilities Education Act, 20 U.S.C. § 1400 (2004).

Lewis, M. M., Burke, M. M., & Decker, J. R. (2021). The Relation between the Individuals with Disabilities Education Act and Special Education Research: A Systematic Review. American Journal of Education, 127(3), 345-368.

Lord, C., Charman, T., Havdahl, A., Carbone, P., Anagnostou, E., Boyd, B., ... & McCauley, J. B. (2022). The Lancet Commission on the future of care and clinical research in autism. The Lancet, 399(10321), 271-334.

Lovass, I. (1996). Criteria for appropriate treatments. Los Angeles: University of California.

MacDuff, G. S., Krantz, P. J., & McClannahan, L. E. (2001).Prompts and prompt-fading strategies for people with autism.

Mesibov, G. B., Shea, V., & Schopler, E. (2004). The TEACCH approach to autism spectrum disorders. New York: Springer Science+Business Media.

New York State Department of Health Early Intervention Program. (1999). Clinical practice guideline: The guideline technical report Autism/pervasive develop- mental disorders. New York.

Peters-Scheffer, N., Didden, R., Korzilius, H., & Sturmey, P. (2011). A meta-analytic study on the effectiveness of comprehensive ABA-based early intervention programs for children with autism spectrum disorders. Research in Autism Spectrum Disorders, 5(1), 60–69. doi:10.1016/j. rasd.2010.03.011

Reser, J. E. (2011). Conceptualizing the autism spectrum in terms of natural selection and behavioral ecology: the solitary forager hypothesis. Evolutionary Psychology, 9(2), 147470491100900209.

Sandoval-Norton, A. H., Shkedy, G. (2019). How much compliance is too much compliance: Is long-term ABA therapy abuse?. Cogent Psychology, 6(1), 1641258.

Smith, T., Groen, A. D., & Wynn, J. W. (2000). Randomized trial of intensive early intervention for children with pervasive developmental disorder. American Journal on Mental Retardation, 105(4), 269–285. https://doi.org/10.1352/0895-8017(2000)105%3c0269:RTOIEI%3e2.0.CO;2.

Sternberg, R. J., & Williams, W. M. (2010). Educational psy- chology (2nd ed.). Columbus, OH: Pearson Education.

Virués-Ortega, J. (2010). Applied behavior analytic inter- vention for autism in early childhood: Meta-analysis, meta-regression and dose–Response meta-analysis of multiple outcomes. Clinical Psychology Review, 30 (4), 387–399. doi:10.1016/j.cpr.2009.10.004

Vivanti, G., Kasari, C., Green, J., Mandell, D., Maye, M., & Hudry, K. (2018). Implementing and evaluating early intervention for children with autism: Where are the gaps and what should we do?. Autism research, 11(1), 16-23.

Wilson, B., Beamish, W., Hay, S., & Attwood, T. (2014). Prompt dependency beyond childhood: Adults with Asperger’s syndrome and intimate relationships. Journal of Relationships Research, 5. doi:10.1017/ jrr.2014.11