Effective Assessment and Progress Monitoring in an Online Environment: A Study in Six Countries
This study sought to understand more about how teachers and school leaders in low and middle-income countries assessed student progress when students were learning remotely during the COVID-19 pandemic.
The study was undertaken in six countries – Bangladesh, India, Nigeria, Pakistan, Philippines, and South Africa. The data collection involved focus group discussions with two held in each country.
It was evident from the focus group discussions that teachers had faced numerous challenges and obstacles in seeking to maintain learning provision for their students during the pandemic. The issue of how best to assess progress appeared a secondary issue to the key priority of finding ways to reach as many students as possible and then engage them in learning effectively.
The innovative approach to Learning
Teacher approach to assessment was heavily interlinked with student access to digital devices and connectivity. Where the infrastructure allowed, many teachers frequently utilised EdTech tools for assessment. Free to use multiple-choice quizzing tools such as Google Forms were widely used and comparatively easy to deploy. Many teachers cited EdTech assessment tools as having a positive impact on student engagement
Many participants explained how they had to use asynchronous learning approaches, as many students had to share access to devices with their parents and/or siblings. In these instances, teachers often sent resources on WhatsApp or sent links to a third-party quizzing tool. A hybrid approach to assessment was needed when students did not have access to a device. Other students’ parents in their community were relied upon to help students submit their work. It was common for students without access to send pictures of their written answers via a friend/friend’s parents’ phone to the teacher. In instances where a digital-first approach was not possible, schools distributed hard copy resources (typically workbooks) via parental collection or by visiting house to house to distribute the worksheets. Teachers often resorted to seeking one-to-one contact, either via phone, recorded voice messages, or as a last resort via physical visits.
For those that could not be reached easily by other means, broadcast radio and TV provided some learning material, but participants in the focus groups were sceptical as to student engagement in the learning and struggled to find an effective means to monitor their Education 4 learning progress. Governments, broadcasters, and education resource providers should consider how they can include interactive elements into broadcast education content to ensure greater student engagement. Scheduled broadcasts could be published with supporting information regarding the lesson topics and learning objectives covered, as well as interactive tasks, potentially via a partner app.
Common Challenges Affecting Virtual Learning
A complicating factor for both students and teachers was the issue of mobile data costs. The study found that the issue of the cost of data restricts how students can learn and limits how teachers are able to deliver lessons and progress monitor. For example, in many low-resource settings, teachers relied on recorded WhatsApp voice messages, asking simple comprehension questions to the student to seek to assess their progress as this form of delivery was most efficient in terms of data costs and suited for asynchronous learning.
Adapting assessment techniques to remote learning resulted in a range of new challenges. A common theme was concern from teachers regarding the authenticity of assessments, with concern students were sharing answers in WhatsApp groups or searching for answers online. Some participants used features in EdTech tools to mitigate this such as shuffling questions and using timed assessments, but suspicions remained. For education systems that have a culture of high stakes assessment focus, the issue of assessment authenticity is a critical one and EdTech providers should ensure, relevant features to mitigate are offered and well communicated.
As teachers sought to ensure learning continuity via whatever means possible, and by extension utilise a range of approaches to monitor their students’ progress, teachers have deployed extremely inefficient workflows, creating huge workload, stress, and often direct financial costs for themselves. Opportunities exist for new EdTech solutions to combine the low cost/free and low data consumption benefits of tools such as WhatsApp, with a more streamlined approach to communication and learning resource management. Some teachers cited the fact that their personal phones quickly ran out of storage after receiving hundreds of messages and photos of work from students.
EdTech solutions, could seek to alleviate this pressure, either by using cloud storage solutions or alternatively using file storage tools that would allow teachers to delete files from their phones after they’ve been simply saved in a safe location elsewhere. In addition, governments and system leaders can undertake simple steps to support teacher workload challenges and reduce the friction around EdTech uptake for example by developing centralised resource libraries of useful EdTech tools.
During the focus group discussions, teachers and school leaders described how they had experimented with different forms of distance learning. It was curious and concerning, therefore, that there was much less experimentation with effective progress monitoring pedagogy. A combination of factors impacted this, for example, the extreme workload faced and the fact that tools offering multiple choice questions were widely available, with preprepared question banks and these tools were typically easy to use, and critically free. The issue of data costs, also restricted format innovation for many. In addition, a lack of dedicated professional development regarding assessment pedagogy took place. Governments, researchers, system leaders and school leaders, should review if sufficient resources, tools, and professional development support exist to support teachers in developing a range of assessments.
For governments and school leaders to identify and seek to address learning gaps that have developed, structured data collection systems are required and could provide significant benefits. The focus groups however, highlighted a widespread lack of school or departmental level strategy and activity regarding how to use and store data/learnings gathered from online formative assessments. A small number of individual teachers provided anecdotes on how they collected, stored and analysed student data, typically to support with instructional differentiation efforts and remediation identification.
The study identified, that whilst EdTech tools have proven popular with teachers as means of developing student engagement and conducting formative assessment, governments and schools in LMICs face many urgent challenges to ensure effective and consistent progress monitoring is possible whilst learning online. If governments are to address learning gaps that have widened during the pandemic, then ensuring all teachers are able to utilise effective techniques for progress monitoring in an online learning environment will be key.
