Evaluating Digital Personalised Learning Tools in Kenya: a new research study in collaboration with EIDU
In recent years, interest in the ways that educational technology (EdTech) can personalise learning has increased. Personalised learning approaches build on pedagogical models which enable students to access content and learning experiences that are matched to their level. Recently, research has begun to focus on the way that EdTech tools can support the learner-centred and flexible benefits of personalised learning, whilst informing educators’ practices through data-driven decision making.
As outlined in a previous blog, there is a growing base of strong evidence on how technology-supported personalised learning can improve learning outcomes. Such tools have been reported to improve learning outcomes, support ‘teaching at the right level’, close educational gaps for lower attaining students, and enhance the role of the teacher.
It is within this context that EdTech Hub is launching a new research study to rigorously evaluate the integration of what we call digital personalised learning (DPL) into Kenyan classrooms for young children, aged between 4-8 years old. The study aims, for the first time, to determine how a DPL tool can most effectively be integrated into classroom instruction to improve learning in a low-middle income country (LMIC). In this blog we introduce the aims and objectives of the study, and highlight its anticipated contribution to research and learning in this field.
What is digital personalised learning?
Broadly defined, DPL can be considered as “the ways in which technology enables or supports learning, based upon particular characteristics of relevance or importance to learners.” Such tools utilise the potential of technology to facilitate learning which is driven by student needs and interests. For instance, DPL tools could be designed to adapt to the student’s pace of learning, or else provide educational content tailored to the student’s interests or current learning level.
Digital personalised learning (DPL) is a broad umbrella term that encompasses a range of approaches. You might have previously heard of a number of other terms, including ‘personalised adaptive learning’ (PAL), ‘computer-assisted learning’ (CAL), ‘computer-aided learning’ (also CAL), ‘computer-aided instruction’ (CAI), and ‘intelligent/cognitive tutoring systems’ (ITS/CTS). While there is much commonality between these terms, not all fall under the umbrella of DPL – there are some subtle differences to each approach which demands consideration.
The distinction between ‘responsive’ and ‘adaptive’ personalised learning systems is a particularly helpful way to understand different types of approach which sit within the ‘DPL’ umbrella. As demonstrated in the graphic below, responsive systems are those that may enable learners to personalise the learning interface, choose their own tailored path through instructional material, or provide some degree of personalised support or feedback. Adaptive systems, on the other hand, actively scaffold learning by adapting content delivery depending on user behaviour or performance. Such interventions may adaptively provide content that matches the level of the learner or modify the pace of instruction.
Digital personalised learning can therefore be understood as encompassing both responsive and adaptive approaches to technology-enabled personalised learning.
How much is known about the effectiveness of DPL?
Most research on DPL tools to date has taken place in high-income countries, with considerable variations found in impact. However, less is known about the effectiveness of using DPL in low- and middle-income countries (LMICs). The EdTech Hub has been hard at work trying to build more insights in this space.
A team from EdTech Hub recently conducted the first meta-analysis to evaluate the effectiveness of DPL in LMICs. It revealed that DPL can have a statistically significant positive impact (a combined effect size of 0.18) on learning, and also established how DPL tools which adapt to learners’ ability levels deliver the greatest impact (a higher effect size of 0.35). Such a finding supports the assumption that DPL can positively impact learning outcomes, and particularly emphasises the high potential of EdTech to utilise adaptive personalisation features to enable learning gains.
A recent Rapid Evidence Review identified another evidence gap: that most existing large-scale research involves ‘supplementary’ uses of DPL during sessions outside of regular instruction (e.g. after school or at lunchtime). There therefore remains scope to enhance knowledge about the effectiveness of DPL when this is integrated into classroom practices by teachers.
Building on this, a recently completed EdTech Hub sandbox with onebillion explored a “hybrid model” which incorporates both supplementary use of a DPL tool as well as classroom integration. Focusing in Malawi, where onebillion had already established itself as a supplementary tool, we learned how important it is to gather perspectives and co-create classroom integration approaches alongside teachers. Based on these consultations, a tentative model has been proposed in collaboration with the Ministry of Education.
So… what are we doing?
The EdTech Hub is launching a two-year research study to address these important evidence gaps, working closely in collaboration with EIDU – an international e-learning provider which offers a DPL platform developed for low-cost Android devices. It will determine how EIDU’s platform – with the capacity to adaptively adjust to learners’ level – can most effectively be integrated into classroom instruction to improve numeracy outcomes in Kenyan schools.
The EIDU platform provides child-focused learning content along with structured pedagogy programmes for teachers and is already well established in Kenya with over 30,000 active users. EIDU is content-agnostic, allowing for any provider to integrate new content onto the platform, and the choice and order of content for each individual child is optimised through personalisation algorithms. Digitised assessment tools enable real-time learning measurement which can be fed back to content providers and researchers, facilitating continuous improvement cycles.
In collaboration with EIDU, the EdTech Hub study will use design-based research (DBR) and a randomised control trial (RCT) to develop and rigorously evaluate contextually appropriate approaches for integrating EIDU’s DPL tool into Kenyan classrooms. The study will focus on:
- The optimum duration and intensity of DPL use during classroom instruction;
- How the adaptive software features of DPL can best support learning in the classroom;
- Cost and resource implications associated with DPL.
The research methodology will follow two phases. The first will feature three cycles of DBR – integrating mixed methods research with sandbox methodologies and A/B/n testing – to facilitate the rigorous design and evaluation of in-school DPL models. This phase will have a particular focus on intervention delivery, including the duration and intensity of student use of the EIDU app, and the role of the teacher in facilitating this process. The second phase of the research will consolidate the learnings of the first through an RCT. During this phase, DPL will be used in a hybrid model alongside existing teaching practices, and data will be collected through EIDU’s existing digital assessment tool and/or paper equivalents. The RCT will investigate issues relating to context, effectiveness and cost, involving approximately over 4000 learners.
This research is well positioned to provide insights into how DPL can help enhance numeracy learning outcomes in Kenya and globally. In Kenya, lessons learned from the implementation of EIDU and associated research outcomes will seek to inform wider sector policy design and practice by providing robust evidence on how best to deploy DPL to support children to achieve better numeracy outcomes. An explicit focus of research on girls, learners of different ages, and learners with different baseline attainment levels will help to generate insights into how to best deploy DPL to support these groups. Through the dissemination of transferrable design principles and evidence of effectiveness in world-leading academic journals, the study will also generate transferrable insights to inform other DPL initiatives and advance sector knowledge globally. Analysis of the costs of different implementation strategies – in connection to learning – will also provide novel insights and inform potential investment in the expanding DPL sector.
If this is something you or your organisation are interested in learning more about or collaborating with us on, please contact either the EdTech Hub or EIDU team.
