Building Resilient Education Systems: Why and how should we monitor hybrid learning?  

The onset of the Covid-19 pandemic in 2020 exacerbated many already existing challenges affecting access to education worldwide. At its peak, over 1.6 billion learners globally were affected by school closures, disproportionately impacting children in low- and lower-middle-income countries who were already facing substantial education challenges prior to the pandemic. With 2 trillion hours of in-person schooling lost due to the pandemic, students in over 80% of countries around the world have fallen behind in their learning.

During this period, we also witnessed the resilience of the education sector. In the midst of mass school closures, almost all countries deployed a mix of options to support remote learning, including online learning solutions, messaging apps, TV, radio, and printed kits in countries where a large proportion of the population had no internet connectivity. However, many countries never implemented monitoring systems, so we do not know the extent to which children were reached with remote learning opportunities or how effective such opportunities were in achieving learning outcomes.¹

In 2022, many countries have now pivoted to a hybrid learning approach, which encompasses both remote and in-person learning with online and offline resources. The rise of hybrid learning means that education stakeholders and decision-makers must ensure that they have the systems and capacities to monitor education delivery and learning across a range of contexts. Monitoring hybrid learning is even more relevant as education providers grapple with the challenges of evolving circumstances and seek to build back better, more resilient education systems. The need to strengthen global capacity to monitor hybrid learning, going beyond traditional monitoring approaches, is, therefore, greater than ever. 

Despite this need, the 2021 UNESCO-UNICEF-World Bank-OECD survey² shows that only one out of three countries has a regular monitoring system in place for hybrid learning. Among low- and lower-middle-income countries, this estimate decreases to one out of four countries. UNICEF is committed to supporting Ministries of Education and education decision-makers to effectively monitor the status and implementation of education delivery and learning, including in emergency situations. To this end, the following resources have been developed. 

1. Conceptual Framework for Monitoring Hybrid Learning Delivery Toward Long-term System Strengthening and Resilient Education Systems
2. Operational Guide for Monitoring Hybrid Learning Delivery Toward Long-term System Strengthening and Resilient Education Systems
3. Monitoring Hybrid Learning: A Short Guide
4. Monitoring Distance Learning During School Closures (through household surveys)

The Conceptual Framework contains a comprehensive list of 163 indicators linked to hybrid learning priority areas. It can be used by Ministries of Education to monitor the needs of learners, staff, administrators, and other key education stakeholders engaged in hybrid learning. Figure 1 below shows the hybrid learning attributes captured in the Conceptual Framework. 

Figure 1. Hybrid learning attributes.

The four attributes are defined below:

1. Characteristics capture key information on the institutions and the hybrid learning programme(s) deployed, as well as the teachers and learners.
2. Reach examines the availability of and access to technology or other relevant resources (e.g., take-home learning packages) to deliver and engage in learning.
3. Engagement measures how learners, teachers, administrators, and parents/caregivers participate in education programming (e.g., time spent by teachers to deliver content, amount of engagement from learners in a hybrid learning initiative). 
4. Effectiveness and quality assurance aim to understand the effectiveness of education delivery and the resulting changes in learning outcomes and skills. 

Of the 163 indicators, a shortlist of 12 indicators essential for monitoring hybrid learning is presented below, a starting point for decision-makers seeking to monitor hybrid learning initiatives.  The Conceptual Framework and indicators are designed to be contextualized and modified based on the needs of each country or local area education authority feeding into their national Education Management Information System (EMIS). 

Shortlist of hybrid learning indicators.

Indicator	Attribute
Teachers with knowledge and understanding of required Information and Communications Technology	Characteristic
Schools/institutions equipped to deliver remote learning	Reach
Enrolled learners with access to appropriate space for home-based learning	Reach
Learners with access to internet connection for education purposes	Reach
Learners with parents/caregivers able to support home-based learning	Characteristic
Learners using world-class digital learning solutions to build skills	Engagement
Teachers who received in-service training in the last 12 months by type of training	Effectiveness and Quality Assurance
Teachers able to deliver hybrid learning	Effectiveness and Quality Assurance
Average time spent by learners in learning activities	Engagement
Learners showing an improvement in learning	Effectiveness and Quality Assurance
Learners showing improvement in digital competencies	Effectiveness and Quality Assurance
Government policies are in place to guide hybrid learning	Effectiveness and Quality Assurance

The Operational Guide compiles additional information on topics including sampling design; data collection, analysis, quality assurance, security and protection; and education management information systems (EMISs). It aims to support ministries of education and key stakeholders around the world to adapt the Conceptual Framework for contexts where hybrid learning may take place. The Operational Guide builds on UNICEF’s extensive work such as the EdTech guide and digital learning toolkit, to provide guidance on the planning, delivery, and monitoring of hybrid learning environments and modalities.

The Short Guide summarizes key points and lessons learned on monitoring hybrid learning from the Conceptual Framework and Operational Guide. It answers questions around data collection, management, analysis, and usage using hybrid learning scenarios.

The Monitoring Distance Learning During School Closures guide is intended to be used by decision-makers monitoring the reach and effectiveness of distance learning modalities, including technology-facilitated and blended learning modalities, through surveys of parents/caregivers, children/students and/or teachers. 

Monitoring distance and hybrid learning is an extremely important task that will help generate insights into what is working and what is not working in education. Further, it will also generate learnings and enable adaptation of future initiatives to ensure that all children are being reached. Lastly, in view of limited financial resources, monitoring will help decision-makers identify cost-effective approaches to remote and hybrid learning. Building upon this portfolio of work, the Conceptual Framework will be piloted in a few countries as a next step. 

These documents can be accessed and downloaded at the link below.

• ¹ For an example of results from monitoring surveys where they were conducted, see https://www.unicef.org/press-releases/repeated-school-closures-due-covid-19-leading-learning-loss-and-widening-inequities 

• ² Calculated from the answers to question 4 of module 10 in the Joint Survey on National Education Responses to COVID-19 (3^rd Iteration).