The Use of Artificial Intelligence in English Language Teaching and Learning: Impact of AI Tutoring on Student Achievement

Background: the evolution of language learning technologies

Language teaching has been reshaped at every stage by the technologies available to it. The audiolingual language laboratory of the 1950s and 1960s promised mastery through mechanical repetition and taped models; the arrival of the microcomputer in the 1980s produced computer-assisted language learning (CALL) and a first generation of software drills, grammar checkers, and simulations (Warschauer & Healey, 1998; Chapelle, 2001); the networked and Web 2.0 era of the 2000s added authentic communication, collaboration, and learner-produced content; and the mobile decade that followed extended practice beyond the classroom through app-based learning at scale (Loewen et al., 2019). Each wave was greeted with the same claim, that the technology would finally deliver instruction adapted to the individual learner, and each wave disappointed that claim in the same way, by treating the tool as sufficient and the pedagogy as secondary.

Artificial intelligence entered this history much earlier than the current debate suggests. Weizenbaum's ELIZA (1966) showed that a simple pattern-matching program could sustain a conversation that users found meaningful, and rule-based conversational agents were tried in language learning from the 2000s onward (Fryer & Carpenter, 2006). In parallel, intelligent tutoring systems (ITS) built explicit models of the learner and the domain, tracking mastery of individual skills and selecting practice accordingly; two decades of evaluation found that such systems produce substantial learning gains and approach the effectiveness of human tutoring (VanLehn, 2011; Ma et al., 2014; Kulik & Fletcher, 2016). What changed decisively in 2022 was scale and openness. Large language models (LLMs) can now generate and evaluate connected discourse, hold a multi-turn conversation, explain an error in the learner's own language, and role-play a job interview or a service encounter on demand (Kasneci et al., 2023; Kohnke et al., 2023). The tutor is no longer a closed item bank with a rule engine attached; it is an interlocutor.

Rationale

English is the most widely studied additional language in the world, and the demand for demonstrable proficiency continues to grow in education, employment, and migration. At the same time, the conditions under which English is taught have not improved proportionally: large classes, limited contact hours, and a syllabus that must move forward whether or not each learner is ready. The result is the oldest unsolved problem in instruction, described by Bloom (1984) as the two-sigma problem: the gap between what group instruction achieves and what one-to-one tutoring achieves. Learners who need the most practice usually receive the least individual attention.

An AI tutor that is always available, patient, unaffected by social pressure, capable of producing unlimited low-stakes practice, and able to respond immediately to what a learner actually writes or says speaks directly to that problem. If it recovers even a fraction of the human tutoring advantage, the implications for equity in English language teaching (ELT) are considerable, particularly for learners who cannot purchase individual instruction.

Adoption, however, is running well ahead of evidence. Reviews of AI in education consistently find that deployment decisions are driven by vendor claims, institutional strategy, and expectation rather than by trustworthy evidence of learning impact, and that the published literature is dominated by short, small-scale, single-group studies (Zawacki-Richter et al., 2019; Crompton & Burke, 2023). Teaching time, learner attention, and institutional budgets are finite; if AI tutoring does not improve proficiency beyond ordinary instruction, those resources are better spent elsewhere, and if it does, learners who are currently excluded from it are being excluded from a real advantage. This study is designed to produce that evidence in a specific, high-demand setting: English language teaching and learning.

Research on artificial intelligence in education is plentiful but unevenly distributed. Systematic reviews show a concentration of work in computer science, science and engineering education, and higher-education administration, with comparatively thin coverage of language learning as a domain in its own right (Zawacki-Richter et al., 2019; Ouyang & Jiao, 2021). Where AI and language learning do intersect, the literature tends to be conceptual, small in scope, and short in duration, documenting enthusiasm and possibility far more often than measured learning (Kohnke et al., 2023; Zhang & Zou, 2022). The consequence is that instructors and institutions cannot answer the question they actually face: does AI tutoring improve English proficiency relative to what we already do, and under what conditions?

Three specific gaps define the problem this study addresses.

• Outcome evidence. Few studies measure proficiency gain against a comparison group taught by conventional means, so the incremental effect of AI tutoring over ordinary instruction remains unclear. The strongest evidence base available, meta-analyses of intelligent tutoring systems, reports moderate to large effects but draws predominantly on mathematics and science and substantially predates generative AI (Ma et al., 2014; Kulik & Fletcher, 2016).
• Engagement evidence. Engagement is typically inferred from usage logs or from satisfaction items written for a single study, rather than measured with validated instruments and reported across its behavioural, emotional, and cognitive dimensions. It is therefore difficult to distinguish meaningful engagement from novelty-driven time on task, a distinction that matters for whether any effect will persist.
• Learner voice. Students' own accounts of AI tutoring relative to human teaching are seldom reported alongside quantitative outcomes, even though those accounts shape whether a tool is used well, when it is abandoned, and where it misleads (Selwyn, 2019; Barrot, 2023).

Taken together, these gaps leave institutions unable to separate genuine effect from enthusiasm, and unable to identify the pedagogical conditions under which AI tutoring works. In the absence of controlled evidence, there is a real risk of adopting systems that increase cost, screen time, and learner dependence without improving proficiency, while the question of how AI tutoring should be integrated with, rather than substituted for, teaching remains open.

The study pursues three objectives, each expressed below as a research question and an associated hypothesis.

1. 1.To analyse how AI tutoring systems can enhance student engagement in English language learning.
2. 2.To evaluate the impact of personalised learning through AI on student achievement in English language proficiency.
3. 3.To investigate students' perceptions of using AI tutors compared with traditional methods of instruction.

Research questions

• RQ1. How does the use of an AI tutor influence students' behavioural, emotional, and cognitive engagement in English language learning compared with conventional instruction?
• RQ2. To what extent does AI tutoring improve English language proficiency, and how are gains related to personalised pacing and immediate feedback?
• RQ3. How do students perceive AI tutoring relative to traditional instruction, and what do they identify as its principal affordances and limitations?

Hypotheses

• H1. Students in the AI-tutoring condition will demonstrate significantly greater gains in English language proficiency than students receiving conventional instruction, controlling for pre-test performance.
• H2. Students in the AI-tutoring condition will report significantly higher engagement, particularly in its behavioural and emotional dimensions, than students in the comparison condition.
• H3. Students will report broadly favourable perceptions of the AI tutor, while identifying feedback accuracy, pragmatic and cultural nuance, and the absence of human interaction as its principal limitations.

Technology in education and the turn to artificial intelligence

Theoretical accounts of technology-mediated learning rest on the claim that learning is a social and mediated process. Vygotsky's (1978) zone of proximal development locates development in the space between what a learner can do alone and what the learner can do with assistance, and treats that assistance, or scaffolding, as temporary and contingent. Applied to language learning, the construct predicts that a tutor's value lies in calibrating support to the learner's current level and withdrawing it as competence grows. The interaction hypothesis (Long, 1996) adds that acquisition is driven by negotiated interaction, in which meaning is clarified and repair occurs; the output hypothesis (Swain, 1995) adds that producing language, and noticing the gap between what one intended and what one managed, is itself a mechanism of learning. Krashen's (1982) affective filter hypothesis adds a motivational condition: anxiety and low confidence can block the uptake of comprehensible input even when it is plentiful.

Read together, these perspectives specify precisely the demands that a language tutor must meet: calibrated difficulty, abundant opportunity for production, immediate and interpretable feedback, and an emotional climate in which practice feels safe. They also explain why technology has repeatedly fallen short. Early CALL was strong on repetition and weak on negotiated meaning (Warschauer & Healey, 1998); app-based learning offered volume and convenience but limited open production (Loewen et al., 2019); ITS offered genuine adaptivity but within fixed domains, such as grammar or vocabulary, where the learner's response space could be formalised (Ma et al., 2014).

Machine learning algorithms and the personalisation of instruction

Personalisation in AI-supported learning is produced by a learner model. In ITS, that model typically estimates mastery of each skill from the learner's performance, using techniques such as Bayesian knowledge tracing, logistic models from item response theory, or, more recently, deep neural knowledge tracing, and it then selects the next item, hint, or explanation to maximise learning (VanLehn, 2011; Ma et al., 2014). Reinforcement learning has been used to sequence practice and to decide when to introduce difficulty, and natural language processing to analyse free responses and to diagnose errors.

Generative models change the mechanism rather than the goal. Instead of a hand-built model of the domain, an LLM represents statistical regularities of language at scale, which allows it to engage with open-ended production, to explain why an utterance is inappropriate rather than merely wrong, and to adopt a role and a register on demand (Kasneci et al., 2023). The practical advantages are considerable: unlimited practice in the productive skills, feedback delivered within seconds, and a marginal cost per interaction that approaches zero. The associated risks are equally specific. Model responses are probabilistic and can be confidently wrong; feedback on pragmatics, register, and cultural appropriateness is the least reliable part of the system precisely where learners have least basis for judging it; and the reasoning behind a correction is not exposed, which makes it difficult for a learner to calibrate trust (Barrot, 2023; Kohnke et al., 2023).

Benefits reported in the literature

• Availability and practice volume. An AI tutor removes the constraint of the teacher's time, allowing substantially more practice than a curriculum alone permits. App-based studies show that additional, well-designed practice produces measurable gains (Loewen et al., 2019).
• Immediate corrective feedback. Feedback delivered within seconds, while the learner still holds the intended meaning in working memory, is more useful than delayed correction, and is consistently associated with improved accuracy (Hattie, 2009; VanLehn, 2011).
• Reduced anxiety and greater willingness to communicate. Because the interlocutor does not judge, learners report practising more, repeating attempts, and asking questions they would avoid in class (Kohnke et al., 2023). This maps directly onto the affective filter (Krashen, 1982) and onto the motivational need for competence support in self-determination theory (Ryan & Deci, 2000).
• Individualised pace. Learners who need longer on a structure, or who are ready to move on, are no longer held to the pace of the group, which is the essential promise of tutoring (Bloom, 1984).
• Scalability and equity. Tutoring support that is not rationed by price or geography has obvious distributional value, provided access to devices and connectivity is addressed.

Challenges and open questions

• Feedback reliability. Generative feedback on the productive skills can be inaccurate, incomplete, or inconsistent between attempts, particularly on pragmatics, coherence, and register (Barrot, 2023).
• Over-reliance and deskilling. Assistance that always produces fluent text can remove the productive struggle through which writing develops, and may mask the learner's actual level from both learner and teacher.
• Data privacy and governance. Language tutoring data includes voice recordings and personal narratives, and the ethics of retaining, profiling, or transmitting such data are only beginning to be addressed (Holmes et al., 2019).
• Equity within the tool. Performance varies across language varieties, accents, and first-language backgrounds, which raises the possibility that automated judgments reproduce existing disadvantage.
• Teacher preparedness. Integration depends on teachers' technological pedagogical content knowledge, not on tool availability; where that knowledge is absent, tools are used as substitutions for existing activities rather than as transformations of them (Mishra & Koehler, 2006).
• Assessment validity. If an AI tutor can produce the work being assessed, both the measurement of achievement and the design of the study must be reconsidered (Barrot, 2023).
• Novelty effects. Early engagement with a new tool is frequently inflated, so short studies risk attributing to pedagogy what belongs to novelty (Zawacki-Richter et al., 2019).

Across this literature, the benefit cases are largely theoretical or drawn from other subject domains, and the challenge cases are largely conceptual. What is missing is controlled, classroom-based evidence on proficiency, measured with validated instruments, in a study long enough for novelty to recede and accompanied by learners' own accounts. The methodology below is designed to supply it.

Research design

The study uses a mixed-methods design with a quasi-experimental core: a non-equivalent comparison group, pre-test and post-test structure, combined with an embedded qualitative strand (Creswell & Plano Clark, 2018). Two intact classes taking the same course will be allocated to conditions, one learning with an AI tutor integrated into the course and one receiving the same instruction without it. Intact classes are used because assigning individual students to conditions would require splitting an existing class and would change the instructional setting the study is meant to examine. The quantitative strand establishes the size and direction of effects on proficiency and engagement; the qualitative strand explains how and why those effects occur, and where the tutor fails.

Participants

Participants will be approximately 120 learners (two classes of about 60) enrolled in an English for Academic Purposes course at a public university. Ages will range from 18 to 24. All participants will have been placed at an intermediate level, corresponding to B1 on the Common European Framework of Reference, by the institution's own placement test. First-language backgrounds will be mixed and gender distribution approximately balanced, reflecting the composition of the programme.

Sampling is purposive at the level of the class, because the intervention is delivered to intact instructional groups; the two classes will be allocated to conditions by random draw, and equivalence at pre-test will be verified statistically. Inclusion requires enrolment in the course, B1 placement, and informed consent. Learners will be excluded from the analysis if they attend fewer than 80 per cent of sessions, if they fail to complete either the pre-test or the post-test, or if they have previously taken a course that used the same AI tutor. For the qualitative strand, 12 to 15 participants will be selected purposively from the AI condition to include high, typical, and low gainers, so that the interview data are not drawn only from the most enthusiastic users.

Intervention and material conditions

The intervention runs for 12 weeks, with two 45-minute sessions per week plus assigned practice. Both conditions follow the same syllabus, use the same materials, complete the same written and oral tasks, and are taught by the same teacher; the only difference is that learners in the AI condition practise with an AI tutor for 20 to 25 minutes in each session and between sessions. The tutor combines conversational practice with corrective feedback, retrieval of previously learned items at spaced intervals, and speech interaction for pronunciation and fluency. Both conditions are assessed identically. Teacher training of one and a half days precedes the study, covering task design with the tutor, interpretation of its feedback, and the distinction between practice and assessment.

Data collection methods

• Proficiency test. Adapted institutional test at pre-test and post-test, comprising grammar and vocabulary, reading, listening, a timed written task, and a recorded speaking task. Writing and speaking will be double-rated by two trained raters using an analytic rubric, with inter-rater agreement reported (target Cohen's kappa of .80 or above).
• Engagement questionnaire. A validated instrument adapted from existing engagement measures, capturing behavioural, emotional, and cognitive engagement on a five-point scale, with internal consistency reported for each subscale.
• Perception survey. Twenty-five five-point items covering perceived usefulness, ease of use, feedback quality, anxiety, and preference relative to traditional instruction, together with three open-ended items.
• Semi-structured interviews. Thirty to forty minutes with 12 to 15 learners from the AI condition, exploring how they used the tutor, what they believed it did for them, and where they distrusted or abandoned it. Interviews will be audio-recorded and transcribed verbatim.
• Interaction logs. Tutor session data, including time on task, number of turns, request types, and error categories, used to describe dosage and to test whether gain is related to use.
• Teacher observation notes. A structured protocol completed after each session to document fidelity of implementation and any deviations in either condition.

Data analysis

Quantitative data will be analysed with descriptive statistics and inferential tests: paired-samples tests for within-group change, an independent-samples comparison of gain scores, and analysis of covariance with pre-test score as covariate to control for initial differences, reporting effect sizes alongside significance. Engagement will be analysed by subscale. Qualitative data will be analysed thematically using an inductive, six-phase approach (Braun & Clarke, 2006), with two coders, a documented codebook, and member checking of the emerging themes with a subsample of participants. The two strands will be integrated in a joint display that sets quantitative outcomes beside the qualitative explanations that account for them.

Ethical considerations

Approval will be sought from the institutional research ethics committee before any data collection. Participation is voluntary, unrelated to course grades, and withdrawable at any point without consequence. Written informed consent will be obtained for participation, for audio recording, and for the use of log data; all data will be pseudonymised, stored securely, and reported only in aggregate or as unattributed quotation. Learners will be told explicitly that AI feedback can be inaccurate and should be checked with the teacher, and teaching staff will be able to see which tasks were completed with AI assistance.

Limitations and delimitations

The design cannot eliminate every threat to validity. Allocation is by class rather than by individual, so pre-existing differences between classes cannot be fully excluded, although pre-test equivalence will be tested and used as a covariate. The study is conducted at one institution over one semester, which bounds generalisability to similar learners and settings. Gains may be partly attributable to novelty, engagement is in part self-reported, and the behaviour of a generative tutor depends on the model version in use, which limits exact reproducibility. These constraints are stated rather than solved, and the discussion will interpret findings within them.

In relation to the second objective and H1, the study anticipates a moderate to large advantage for the AI-tutoring condition on the composite proficiency measure, with the largest differences on the productive skills. Writing and speaking are the skills that depend most on the volume of practice a learner can obtain and on the speed of feedback, which are precisely the constraints an AI tutor relaxes and a conventional curriculum does not. Differences in reading and listening are expected to be smaller, because those skills receive substantial exposure in regular instruction already.

In relation to the first objective and H2, engagement is expected to be higher in the AI condition, most clearly in its behavioural dimension, reflecting greater time on task, and in its emotional dimension, reflecting reduced anxiety about making mistakes in front of others. A decline in usage across the second half of the intervention is also anticipated; if it appears, it will be reported rather than smoothed over, because it bears directly on whether engagement gains are durable or novelty-driven.

In relation to the third objective and H3, learners are expected to describe the tutor as useful for practice, confidence, and immediacy of feedback, and at the same time to report distrust of its explanations, difficulty with idiom, register, and cultural nuance, and a continuing preference for the teacher's judgement. This combination is not a contradiction; it describes a division of labour, and it is the finding most likely to change how institutions deploy such tools.

Anticipated contribution

The study contributes controlled, classroom-based evidence on proficiency, engagement, and learner perception in a domain where such evidence is scarce, and it does so with instruments that other researchers can reuse. More practically, it yields the implementation conditions that determine whether AI tutoring works: the dosage, the way practice feeds back into instruction, and the role the teacher retains. If the anticipated gains are not observed, that outcome is equally decision-relevant, because it argues against displacing teaching time with a tool that does not improve learning, and would redirect attention to the pedagogical conditions under which such tools might.

Artificial intelligence is the most consequential new variable in English language teaching of the past decade, and the evidence base for it has not kept pace with its adoption. This proposal set out the development of language learning technologies from the language laboratory to conversational generative models, identified three gaps in the current research concerning outcome evidence, measured engagement, and learner voice, and defined three objectives addressing engagement, achievement through personalised learning, and student perception.

The methodology described here, a twelve-week mixed-methods study with a comparison group, validated measures, and interviews with learners, is intended to close those gaps sufficiently for instructors and institutions to decide on evidence rather than expectation. Its premise is that the value of AI tutoring lies not in replacing the teacher but in extending the practice that only individual attention has so far been able to provide, and that the question worth answering is not whether AI can tutor, but whether its tutoring improves what learners can actually do with English.