Learning in the Post-COVID Era: What Makes Us Build Back Better
The education system in Pakistan has seen remarkable progress, with primary enrolment increasing from 55% to 73% in less than two decades. However, challenges remain, as average student learning needs significant improvement. About 44% of Class 3 students cannot read a simple sentence in English, highlighting the ongoing need for focused efforts to enhance educational outcomes (Annual Status of Education Report (ASER), 2021). Although the pandemic exacerbated the situation globally, Pakistan was hit extremely hard in the early years. The adverse effect of Covid-19 on learning is expected to be compounded by frequent school disruptions due to natural disasters (such as the 2022 floods) and other unforeseen circumstances.
During our visits to schools across Pakistan to assess literacy and numeracy skills, we observed Class 4 students in a government school of Pakistan’s Khyber Pakhtunkhwa (KP) province as the teacher asked them to read the sentence “I come to school every day and learn new things”. While only one in twenty students could successfully read the sentence, others made a commendable effort when the teacher broke the sentence into smaller parts. Some students interpreted ‘everyday’ to mean ‘new’ and others pronounced ‘come’ as ‘chum’, shedding light on the specific literacy skills at the primary school level.
In our discussions with teachers from public and private primary schools across Pakistan, we observed that children had fallen even further behind their grade levels when they returned to school after Covid-19 closures. They not only learnt less during the Covid-19 closures but, they also likely needed to remember what they had learned prior to the closures. To complicate matters, an increasingly predominant feature of post-covid classrooms is higher within-the-classroom inequality, possibly leading to student disengagement, drop-outs, and a strain on already overburdened teachers.
To this end, our Learning and Education Achievements in Pakistan (LEAPS) research team, supported by EdTech Hub and other international and local partners, has been implementing and evaluating technology-enabled targeted instruction programs, focused on foundational learning, across public primary schools in Pakistan.
What does it take to ‘build back better’?
The pandemic’s learning shock is a great opportunity to build the Pakistani education system back better and address the broader learning crisis. One of the most important components of successfully ‘building back better’ the public education system is political buy-in.
In our experience, the Ministry of Federal Education & Professional Training (MoFE&PT) and the Khyber Pakhtunkhwa Elementary & Secondary Education Department (KPE&SED) have been instrumental in unequivocally shifting focus to prioritisation of learning recovery broadly, rather than just responding to the one-time covid shock. The government’s willingness to test innovative learning-focused interventions was a pivot to opportunity for research teams such as ours.
To address the learning crisis, the federal and provincial governments endorsed a large-scale rollout of our post-covid remedial foundational program, the Targeted Instruction in Pakistan (TIP), which we implemented in public primary schools across both the Islamabad Capital Territory (ICT) and Khyber Pakhtunkhwa (KP) province. TIP uses a systems approach to target groups of actors in the education ecosystem: students, teachers, head teachers, and parents.
Our systems thinking approach was first deployed as part of over two decades of our research under LEAPS. It understands that learning is a complex process involving various actors interacting with one another. TIP uses technology to help these actors and improve the efficiency of their interactions. While our KP study focuses on teachers who use technology to help students learn better, our ICT study targets parents and other home-based actors as well, with an emphasis on measuring the impact of technology on learning.
For students, TIP helps them catch up to grade level by plugging gaps in their knowledge, using specially designed materials and technology. In the case of teachers, TIP’s tech tool makes repeated student testing easier and enables teachers to tailor instruction to individual student learning needs more efficiently; a strategy otherwise practised by teachers in an ad hoc manner.
Similarly, TIP’s focus on a foundational curriculum allows teachers to move away from an overambitious curriculum that only teaches to the test. In ICT, head teachers monitor teacher and student progress more effectively through access to the same tools and data teachers have. Likewise, parents and caregivers – who may otherwise be disengaged from schools – are nudged to involve them more in their child’s learning.
What did we learn about how much students are learning?
This past year, the LEAPS team conducted massive diagnostic testing in English, Urdu, and math with approximately 280,000 male and female students in public primary schools, from grades 1 through 5. Testing in Khyber Pakhtunkhwa (KP) covered almost 205,000 students from 1,250 government schools, while in Islamabad Capital Territory (ICT) 77,000 students in 560 government schools were tested.
Our specially designed tests measure learning gaps and sort students into learning groups for each subject based on the student learning objectives (SLOs) taught in previous grades. These SLOs are borrowed from the national curriculum and the ASER learning assessments. Such diagnostic testing identifies students who are behind their grade level and thus require remediation. Teachers were involved in the administration of diagnostic testing to instill a sense of ownership amongst them and gain their buy-in for TIP implementation in schools.
At first glance, the diagnostic results may appear bleak. Looking closely, they offer an in-depth diagnosis of covid-induced learning losses in government schools, paving the way for a system-level overhaul of the public education landscape.
1. Most students are desperately in need of remediation
More than half the students in every subject and class need remediation, suggesting that coronavirus likely compounded pre-existing learning losses. In KP, nearly half of Class 1 is already behind their grade level in literacy and numeracy. By Class 2, this increases to between 80% and 85%. Classes 3 and 4 hover around 90% for math and English, although they remain around 80% for Urdu. Interestingly, there is an unexpected drop in the need for remediation in Class 5 where students, in the range of 60% to a little over 70%, perform below their grade level. Although there is a little more by-subject variation in ICT than in KP, the overarching need for remediation remains similar across both regions.
2. Urdu is the strongest subject
Students across almost all grades and both regions performed better in Urdu than in English and math. This would sound logical given that the country’s national language is Urdu, so children likely get to learn and practice that outside of school as well. However, it is surprising that this trend holds in both KP and ICT, despite KP being a Pashto-speaking province where the language of instruction (and likely at home) is primarily Pashto.
3. Class level is not a determinant of the extent of remediation needed
Despite some of the trends described above, there is no linear relationship between class level and learning level. In KP, besides Class 1, Class 5 students are the least in need of remediation in any of the three subjects. We conjecture that these students undergo some form of remediation in preparation for centralized exams, starting Class 4. These students still have not mastered the basic foundational skills required before graduating from primary school. For example, 73% of Class 5 students cannot write simple paragraphs in English, while 68% of the students cannot read or understand simple paragraphs.
By strong contrast, Class 5 students in ICT are at least as (if not more) behind their previous grades. In math, of the 90% in need of remediation: approximately 60% of students are one level behind, while the remaining 30% of students are two or more levels behind.
4. Girls show slightly less need for remediation than boys
There is no substantive gender-wise variation in student learning distribution. In KP, however, girls show slightly less need for remediation compared to boys. On average, about 5 percentage points more boys need remediation than girls across all grades and subjects, although the gap appears to widen as students’ progress over grades. This difference on average appears to be largest in the languages and smallest in math.
In ICT, the average gap appears to be a little larger, closer to 10 percentage points between the performance of girls and boys.
5. Within versus across the region and school variation
While urban areas need slightly less remediation than rural areas in both ICT and KP, this trend varies on a school-to-school basis. In the same circle or zone (the smallest administrative unit above the school), you can find both schools that need aggressive remediation across all grades and subjects and schools that require minimal remediation. It is, however, not the case that some low-performing schools are causing a high need for remediation. This variation is far more evident within the school where students in one class level may be far behind their grade level compared to students in another.
Similarly, such variation can also be found in different sections of the same class level. In 40% of math classrooms, students are at three different grade levels in terms of their understanding of the material. For example, a 3rd grade teacher often has a mix of students who are a 1st grade, 2nd grade and 3rd grade math level. This spread in students’ learning levels within the same classroom is understandably very challenging for teachers to manage and thus warrants remediation to be planned at three levels – student, class, and school.
Where do we go from here?
There is an urgent need to counteract the learning crisis. In countries such as Pakistan, this means more than just responding to the covid-19-induced learning losses, but also acknowledging that this has been a long-standing issue. TIP is one of many initiatives launched in post-covid Pakistan, with the backing of government partners, to improve student learning.
Going forward, we will be evaluating whether the innovative use of low-cost technology, by various actors on existing devices, can help boost student learning through the TIP programs underway in ICT and KP. Rapid diagnostic testing and accurate information on test scores, coupled with adequate guidance on teaching practices, can assist teachers and parents in filling in children’s learning gaps. We hope to unpack the potential impact of involving head teachers, parents, and other actors in this learning process.
Footnotes:
¹ “Other international and local development partners” are mentioned in the acknowledgement note at the end of this blog.
Acknowledgements: The Targeted Instruction in Pakistan (TIP) program and study, implemented by the Centre for Economic Research in Pakistan (CERP), is led by researchers from Pomona College, Harvard University, the University of California at Berkeley, the University of Chicago, and The World Bank. We thank our funders at the EdTech Hub, the World Bank (SIEF, RRREP, PREP), the Research on Improving Systems of Education (RISE) programme, Douglas B. Marshall Jr. Family Foundation, and the Ministry of Federal Education & Professional Training (MoFE&PT), for funding TIP in ICT and in KP.
