Scaling personalised learning technology in Malawi
Partner: onebillion
Technology to support personalized learning holds huge potential to address the global learning crisis. However, there is still much to learn about the most effective use and scalability of personalised learning in lower and middle-income countries. onebillion – an EdTech non-profit organisation – and their onetab solution are working to contribute evidence and learning to this discussion.
onebillion provides educational hardware (onetab, a low-cost tablet device) and software (onecourse) designed to deliver literacy and numeracy instruction to children in their own language, at their own level. In Malawi, onebillion works with the Ministry of Education (MOE) and VSO Malawi to provide over 150,000 children regular access to onecourse through the public school system under the Unlocking Talent Initiative, and recently, introduced onetab to 650 families to support children’s learning at home in response to COVID-19 school closures.
With increasing interest from the government and other partners to expand access to onetab in Malawi, onebillion partnered with the EdTech Hub to run a 3-month sandbox focused on exploring the opportunities and barriers associated with scaling – the findings of which would inform a proposal to the Malawi government about how to take onetab to national scale. Sandboxes provide partners with tools, support, and access to evidence to scale promising EdTech interventions. This blog highlights key takeaways from the process and offers suggestions for other EdTech implementers going forward.
What we've learned
From the beginning, we knew the onetab technology itself had the ability to improve learning outcomes, which allowed us to investigate key questions related to the implementation model and delivery at scale – this is what our lessons are about.
Adapting implementation models to work at scale often means changing the role of teachers and caregivers, so getting their input was important early on.
Both onebillion and the EdTech Hub are firm believers that effective use of EdTech is only possible when we design implementation models with the “human element” in mind. In this case, not only those who would be using onetab (children) but also those who could facilitate and support their use in different settings (teachers and caregivers).
To date, onebillion and VSO have piloted several implementation models in Malawi, including:
- The current “extraction model” in which students are pulled out of class to use onetab in a separate learning centre located on the school campus
- The home-based model that provides onetabs to households to facilitate children’s learning at home
When we began the sandbox, onebillion already had some sense of the opportunities and challenges associated with these models. For example, they knew the “extraction model” could pose challenges at scale since the construction of learning centres was expensive, and pulling students out of class could be disruptive. And evidence from the household pilot showed encouraging rates of engagement with onetab among children at home, which might increase with more substantive caregiver support.
Given these findings, onebillion was also interested in exploring a hybrid implementation model in which onetabs would be integrated into the classroom and used at home. But this hybrid model would present its own opportunities and challenges. So, what is the optimal implementation model at scale? Specifically, how should onetabs be used in school and at home, and what types of support will teachers and caregivers need to facilitate effective learner engagement? To answer these questions, we needed to learn from the intended users of the technology.
Sometimes, program designers assume they know the behaviors and challenges of their users, but we wanted data to inform our design. We created a survey to better understand caregivers’ behavioral barriers to higher engagement and any existing daily habits into which onetab could be incorporated. Instead of assuming that teachers could and would integrate onetab into their classroom teaching, we conducted a series of interviews with teachers familiar with onetab through Unlocking Talent. We learned that while teachers were excited about the potential of using onetabs in the classroom, they also expressed concerns about their ability to supervise use in large classes effectively. This information will now inform future workshops with teachers to co-create the specific “use model” in the classroom and the ongoing support needed for teachers.
The sandbox was a great opportunity to leverage international expertise in our journey towards national scaling. There is already a huge evidence base behind this [onetab] solution, and now there is also a robust plan for scaling it to reach every single child, which builds on the existing infrastructure within Malawi and factors in iteration and adaptation to make sure it delivers for the child, the caregiver, the teacher and the Ministry.
ONEBILLION STAFF MEMBER
Cost-effectiveness is a significant consideration at scale and must be factored into the design of implementation models.
We know cost-effectiveness is important. For donors, it enables them to compare interventions. For implementers, it enables them to identify ways to adapt their intervention to make it more financially sustainable.
In the sandbox, we tested whether there was a way to make onetab cost-effective to scale nationwide in Malawi. To do this, we costed out the three implementation models that onebillion was considering – onetab use in learning centres, classrooms, and households – at a national scale to evaluate comparative cost-effectiveness in terms of learning outcomes and establish required funding from donors for the proposed model. These models included costs associated with purchasing and maintaining hardware and staffing costs for coordinators and trainers who would be involved in implementation, among others. And, like all good cost models, they were estimates that we will refine as we learn more.
The cost of the “extraction model” was around $30 per year per child, which although it provides significant learning gains, was too expensive to scale nationwide. Operating in schools without a learning centre costs $7 per child per year, and operating at home costs $6 per child per year. What’s more, time on task using onetab – per child – was shown to be 12 times higher at home compared to other models. If we are able to show similar impact at this reduced cost, this provides a basis for nationwide scale. Two routes to improving cost effectiveness of the onetab intervention are: to increase the number of children per device, and to increase the lifetime of the hardware through effective maintenance.
onebillion now plans to workshop these findings with potential donors to better understand their expectations around cost-effectiveness and adapt the proposed model accordingly.
When identifying a suitable supply chain for national scaling, leverage existing expertise and infrastructure in the country.
The supply chain for onetab distribution was a significant area of uncertainty for the onebillion team. What is the best way to get tablets from the manufacturer to the Ministry of Education to remote households, quickly and securely, at scale? How can we ensure onetabs are sustainably used once they are delivered?
We began exploring these questions by speaking with experts who had wrestled with similar questions in the past. For example, we spoke with Madalo Khoza, who led the UNFPA’s work on Malawi’s 2018 Census, executed using 15,000 tablets. She helped us understand the use of serial numbers and spreadsheets to track and monitor tablets, as well as the use of one supervisor per 6 tablets to keep them secure and maintained. We also spoke to Esnart Chapomba from the Directorate of Basic Education, who provided an insight into how exams and textbooks are distributed in Malawi. We then used these insights, and lessons from onebillion’s existing work, to outline our most critical assumptions around logistics at a national level and our best hypothesis for navigating each one.
These activities ultimately helped us build a preliminary supply chain model that the onebillion team can now use to plan their first deployment of onetabs. It includes MOE-provided data on target schools and the number of children, approaches to transport types, security, tracking, tablet repair and maintenance, and leverages existing infrastructure that the Ministry of Basic Education uses to distribute textbooks and assessments.
Government buy-in is essential for successful scaling but requires early and consistent engagement with the right stakeholders.
From the outset, onebillion and VSO Malawi understood that their ability to scale onetab in an impactful and equitable way was highly dependent on government buy-in. But how do we achieve government buy-in? What are the best ways to meaningfully engage the government in the sandbox and longer-term scaling process?
Consultations with those working in Malawi and Hub experts helped us understand that onebillion’s scaling plans needed to ensure that scale-up of onetab would feed into Malawi’s education strategy and other existing in-country initiatives.
We tried a few different approaches to engage the government during the sandbox. First, we mapped relevant stakeholders to ensure we included those whose roles were most pertinent to onebillion’s work. We then sent a weekly highlights email to keep key stakeholders apprised of our progress and learnings from the sandbox activities. Lastly, we invited key members of the government to review and provide feedback on the proposal for national scale-up. Through these activities, the onebillion team identified a key point of contact within the government that can provide ongoing guidance on scaling from a government perspective and champion onebillion’s efforts within the broader Ministry of Education.
Looking forward
While we learned a tremendous amount through the sandbox, we acknowledge that this was just the beginning of onebillion’s scaling journey. As a final reflection, we want to emphasize that EdTech implementers can (and should) build structured experimentation and learning into their scaling plans. Integrating rapid feedback loops and adaptation into project activities – similar to what we undertook in the sandbox – should not end once a pilot is shown to achieve desired outcomes. There will be many new questions to answer, decisions to make, and challenges to face as personalized learning technology is introduced into new contexts and to new end users.
For onebillion, this meant outlining several key research questions to explore as part of their scale-up process for onetab in Malawi, building off the lessons discussed above. The onebillion team, in partnership with VSO Malawi, now aims to feature these questions – along with related experimentation activities – in their scaling proposal to the Malawi Ministry of Education.
