New Brief: Monitoring Distance Learning During the Covid-19 Pandemic
In the initial rush to respond to school closures, countries focused on rolling out distance education. Policy-makers now realise that effective distance education requires robust data generated via high-quality monitoring. This blog highlights the findings of a brief synthesising effective practices in monitoring distance education.
Since March 2020, the Covid-19 pandemic has impacted 36 million learners in Bangladesh. Schools closed down to prevent the spread of the virus. Distance learning strategies using print, radio, television, and online programmes were implemented to provide educational continuity. While Bangladesh quickly pivoted to distance education approaches, there has been little focus on implementing monitoring systems that help us to understand the success of these projects.
The delay in rolling out systems to monitor distance education is familiar. With limited time and resources available, most countries focused on ensuring children could access education. Now that countries are rolling out distance learning, policy-makers are focusing on how they can gain insight into what is or isn’t working, to ensure that they use both the money and time needed to implement distance learning effectively.
The UK Foreign, Commonwealth and Development Office in Bangladesh asked the EdTech Hub to help it better understand effective practices in monitoring distance learning. In response, we developed this brief synthesising global distance education monitoring practices.
How is distance education being monitored globally?
In researching this brief we found that monitoring systems often only focus on one part of the distance learning equation. In order to get a full view of what is taking place, we suggest an approach that considers four elements: Availability, Access, Usage, and Learning.
Availability: to what extent are distance education materials offered?
A great way to understand availability is to develop a detailed skills taxonomy, which maps the available content to the curriculum. This identifies which topics are taught, which learning outcomes are covered, how many content items exist and the content items’ format.
Access: can students access the materials on offer?
For example, when Teach for Pakistan (TfP) wanted to understand how students were able to access resources they began by making announcements at mosques and gathering contact information from local shopkeepers in an effort to reach learners. Once they had student contact details they conducted phone surveys to better understand their preferred access mechanisms.
Usage: are learners engaging with the materials offered?
In Mali, a set of phone surveys showed that some children were listening to radio lessons that were not appropriate for their age, and some caregivers did not know lessons only took 30 minutes a day. The results of the phone surveys were used to adjust the radio lessons.
Learning: are learners actually learning?
This measures how well learners are grasping content. In Botswana, learning is monitored through assessments over the phone. The surveys showed that primary school children learning numeracy learned best through a combination of SMS messages and phone calls.
Applying the Practice
After analysing the elements of distance learning, we explored how to effectively monitor distance education in Bangladesh. We zoomed in on three dimensions in monitoring distance learning: What to collect, How to collect it, and How to use it.
What to collect
Data collected from a sample of the community rather than each individual child is faster and more cost-effective and will allow you to assess the efficacy of a distance education programme.
How to collect
Many tools can be used to collect data. Tools like community-based assessments, third-party data, SMS surveys, phone surveys, and online quizzes are all ways to collect data. See the diagram below for more ideas.
How to use it
After the data is gathered, it can be used to draw conclusions and make adjustments to projects through an ongoing cycle which includes the following steps:
- Collect data
- Gather and clean data
- Analyse data
- Define action points
- Adjust intervention
Recommendations
The brief concludes with a set of nine recommendations for monitoring distance education in Bangladesh. We will leave you with four of these recommendations which we think are equally applicable to all countries delivering distance education programmes.
1. Determine what data is required to make strategic decisions
Not all data are helpful and some data cannot realistically be collected. Before countries begin to gather data, they should identify what specific data is required to make decisions.
2. Use all relevant data but be aware of limitations
With schools closed, obtaining data is difficult. We need to make sure we make the most use of any data that we get our hands on to inform decision making. However, data sets have limitations. They can be incomplete or biased towards a certain group. Limitations cannot be avoided, but we need to be transparent and make decision makers aware of them.
3. Think beyond traditional data collection approaches
Traditional data collection takes time and is costly. In the current situation, countries need to move quickly to collect data fast. Using a variety of novel approaches can help gather information quickly and will paint a more complete picture of reality, allowing crucial decisions to be made fast.
4. Leverage cross-sectoral synergies for monitoring
Education actors are not the only ones looking to quickly gain access to data to inform decision making. Actors across a wide range of sectors are all in need of up-to-date insights to inform decision making. There is the potential for education actors to create economies of scale by collaborating with personnel from other sectors with similar aims.
