Paper review

A dataset of 1.7 million mathematics interactions includes question text, chosen answers and curriculum alignment. It makes richer student-modelling evaluations possible than identifiers and correctness labels alone.

Weak zero-shot LLM results in these experiments do not establish that every LLM-based student model will fail.

Seven measures grounded in learning science compare simulated students with 1,529 real Eedi tutoring dialogues, examining errors, language, behaviour and knowledge.

Prompting and fine-tuning can improve particular measures without producing a valid substitute for real learners.

On 512,000 Eedi interactions, specialist models predicted the next answer more accurately and cheaply than the general LLM configurations tested.

The comparison concerns this dataset, prediction task and set of model configurations; it is not a ranking of every possible approach.

A discrete-mathematics course compared access to a Socratic chatbot and structured proof feedback. The study found modest homework benefits but no exam gain.

Heavier chatbot use was associated with worse exam performance; that association alone does not show the chatbot caused it.

Two experiments with debugging hints found more substantive reflection when students reflected before receiving a hint. Students nevertheless rated those hints as less helpful.

Problem-solving performance did not differ. The study did not directly test whether preference training harms learning.

In a short logic-tutoring study with 113 undergraduates, both a Bayesian knowledge-tracing policy and a deep-RL policy outperformed random selection of worked examples.

There was no significant overall difference between the two adaptive methods. Reward design and the short study setting matter.

A seven-week trial in five schools compared static hints, human tutoring and LearnLM drafts reviewed by human tutors. Tutoring helped students correct mistakes more often than static hints.

This was a human-supervised system. Transfer results do not establish that an autonomous AI tutor is equivalent or superior to a human tutor.

Two experiments tested an AI tutor alongside deliberately imperfect AI peers. The mathematics experiment found better performance with peers than with the tutor alone.

The participants were adults in short online experiments; time on task and transfer to school settings remain important questions.

MATHia logs and Zoom transcripts from 191 middle-school students show higher problem-solving activity during brief visits from a human tutor.

The observational result measures engagement, not a causal improvement in learning.