Tricking, Not Teaching: Concerns Over Gamified Tutoring Tools
A rather disturbing recent article from the Hechinger Report discusses a study that tested using a “talk meter” during online math tutoring sessions to encourage students to participate more. The talk meter uses AI to calculate the ratio of tutor vs student talking time and displays it on the screen, going from red (low student participation) to green (high participation).
Over 700 tutors and 1200 students were randomly assigned to three groups: one where only tutors saw the meter, one where both tutors and students saw it, and a control group without the meter. When students also saw the meter, they increased their talking time by 18%, especially introverts. The article noted that students saw it like a game or competition to talk more. The talk also focused more on expressing mathematical reasoning, rather than just filling silence.
However, the study’s poor design means we don’t know if math achievement actually improved. Students who saw the meter also got information on the benefits of talking more, so we can’t isolate the meter’s impact. Some students and tutors also found the meter distracting or pressurising at times.
The ‘researcher’ is now working on a talk meter app to encourage participation in classrooms as well. But she acknowledges ‘unintended consequences’ are possible with such measurement tools.
Ah, um …
The goal of improving student maths achievement is not clearly demonstrated by this study design. Could there be financial motivations behind developing and promoting the talk meter technology? A few thoughts on this:
The researchers claim the aim is to boost student participation, which prior research suggests aids learning. But the study does not measure if this translated to better math outcomes.
As an ed-tech product, the talk meter does have commercial potential if marketed to schools and tutoring services. The researchers likely have an interest in demonstrating its impact.
More data on student learning outcomes would be needed to truly evaluate if this tool improves maths achievement. The current study is limited in isolating the talk meter's effects.
So, the purpose and efficacy of the tool in boosting learning is undemonstrated. And selling the technology could be influencing the promotion of the tool in this space.
Behavioursim much?
From talk meters to merit pay, behaviourist fixes rooted in metrics and competition seem the go-to for neoliberal education reformers. This philosophy has infected research, with academics pursuing quantitative studies to earn funding and further careers. But why do these carrot-and-stick methods persist when long repudiated by learning experts?
In truth, they align neatly with the marketised ideology Western education systems now operate under. Standardised testing regimes allowed education to become a marketplace. School choice, league tables, and ever-rising academic key performance indicators provide metrics easily converted into profits and losses. Here behaviourism thrives.
Layer in privilege preservation, and behaviourist tools like talk meters maintain power dynamics too. Force-feeding curriculum engagement preserves the status quo your parents benefited from. No danger of student-centeredness upending traditional disciplines prized by elite institutions.
We must challenge education’s obsessive embrace of behaviourism through critical pedagogy and teacher empowerment. If learning professionals lead reform rooted in equity and development psychology, we may shift the system’s destructive aims. Our young people’s wellbeing and untapped potential depend on us ending the neoliberal metric machine grinds all within.
Our dystopian future?
Talk meters today, but tomorrow brings classroom leaderboards broadcasting students’ statistics in real-time. Before you know it, schools add publicly-facing scoreboards of top participating students whilst principals compete on district and even statewide engagement league tables. Education morphs into a dystopian reality show where your self-worth hinges on measurable interactions and children are reduced to data points. Then the Education Dept. launches curriculum contributor ratings as classroom observations become livestreamed gamified entertainment. Soon a generation of students excel at gamification but crumble without scores, metrics and rewards—actual learning left tragically secondary. Beware opening the engagement analytics Pandora’s box; our young people deserve better than behaviourism-driven education for the Trumans of our 21st century show.
But more to the point …
Back to the study … what’s missing is the fact that a talk meter tool that places pressure on students to participate more could negatively impact or disadvantage students with certain special learning needs. Some potential issues:
Autistic students (AUT IEP eligibility) may find the real-time monitoring and feedback stressful or overstimulating, provoking anxiety. Explicit timers and metrics do not match autistic communication preferences.
Students with interest based nervous systems like ADHD / AuDHD (OHI IEP eligibility) often struggle with impulse control and managing external distractions. A talk meter could add performance pressure unhelpful for their executive functioning.
Students with learning disabilities (SLD IEP eligibility) may require more processing time to articulate mathematical reasoning. Talk metrics could frustrate struggling students if used punitively.
High-anxiety students may also find the perceived performance monitoring and judgment of how much they “should” be talking stressful. This could limit rather than encourage participation.
So whilst the intent is to spark participation, such timer tools risk being counterproductive for neurodiverse students and / or those with high anxiety. Students with IEPs often thrive better with alternative forms of positive reinforcement for participation rather than numeric metrics / pressure.
Thus, a major shortcoming highlighted by this study is the failure to examine the impact of so-called universal education tools on students with disabilities and other special learning needs. Neither the development of the talk meter technology nor the research design apparently considered students protected under IEPs or ELL / EB students requiring language accommodations.
The sample of students is described merely as the tutoring company’s existing client base (known as a ‘convenience sample’). There is no indication the researchers intentionally included or tracked results for neurodiverse students, those with learning disabilities, or non-native English speakers. This mirrors a wider issue – such marginalised student populations are often excluded from or not centered in education research.
So whilst the tool may have motivated more participation from introverted or typical students, we have no data on effects for those requiring inclusive learning supports. Students with IEPs may find explicit talk metrics highly distracting or anxiety-inducing. And ELL / EB students can demonstrate mathematical reasoning without high verbal fluency.
Focus on the why, not the what
Rather than gamifying engagement through questionable means like talk meters, researchers should explore why today’s students increasingly switch off from their set curriculums. And such inquiry must give voice to marginalised student populations excluded from most education studies.
We need to understand 21st century students’ perspectives – including disabled students, English language learners, emerging bilinguals, and those from disadvantaged backgrounds. What motivates and interests them? How do they best learn and express that learning? What barriers do they face in engaging with rigid, academic curriculums designed without their input?
Gamification gimmicks ignore how forced participation breeds resentment, not willing engagement. They also frequently disadvantage vulnerable student groups in the rush to quantify academic performance for all. Instead of doubling down on compliance, we must investigate how to nurture the talents of all students on their own terms.
Inclusive, qualitative research focused on students’ needs could reveal ideas for more flexible, relevant curriculums and student-centered assessment models. Teachers also need support to transition to more creative pedagogies that spark self-driven participation. If we listen, today’s students can guide the way to engagement that maximises their future life chances while understanding better how all young minds work. But first researchers must prioritise amplifying unheard voices.