Unlocking the Power of Evidence-Based Approaches in EdTech Interventions: Insights from Entrepreneurs and Practitioners

“There’s a tendency to assume that gathering data is for researchers, not for start-ups or implementers. We’re here to say that it’s actually everyone’s job!”

– Asad Rahman, Experimentation Lead, EdTech Hub

With the above quote in mind, EdTech Hub hosted a community showcase on 5 April 2023, bringing together practitioners and entrepreneurs from EdTech Hub sandboxes and the EdTech Evidence for Entrepreneurs Course. 

EdTech Hub’s sandbox and entrepreneur community features professionals from diverse environments and contexts: the event was an opportunity to showcase a variety of evidence-based approaches to EdTech interventions, inspire our community, and promote knowledge-sharing and peer learning. 

With presentations by William Nguru, founder and Chief Product Officer of the online learning platform Kurasa (more on this below) and Rabia Tanweer, Learning Lead at EdTech Hub and part of the EdTech Hub Bandarban Sandbox, participants delved into a rich conversation about gathering and applying evidence to their EdTech interventions.

In this blog post, we will explore the emerging themes from the event on the impact of evidence-based approaches across different environments and summarise the key themes and takeaways.

Promoting effective evidence-gathering in learner assessment methods in Kenya

William Nguru is an alumni and micro-grant award recipient from EdTech Hub’s EdTech Evidence for Entrepreneurs course — a course developed and run by EdTech Hub to help EdTech entrepreneurs and intrapreneurs ensure their products leverage the best available EdTech evidence in their development. William took us through the vision and achievements of Kurasa.

In its early days, Kurasa was called Ziada, an online learning platform supporting children during the Covid-19 pandemic. The team then focused on enhancing record keeping and formative assessments, partnering with teachers and schools, and pivoting to the platform now called Kurasa.

Current learner assessment methods in Kenya’s new competency-based curriculum (CBC) have limitations that affect teaching and learning outcomes. To address these issues, Kurasa provides a web-based solution that scaffolds the entire curriculum, allowing teachers to assess learner outcomes in a timely way, prepare lessons, get live analytics, and send weekly reports to parents. Kurasa has proved that this approach leads to better pedagogy and instruction methodologies, more curious and interested learners, better learning outcomes, and productive and motivated teachers and learners.

Since its inception in May 2020, Kurasa has reached over 10,000 learners and 500 teachers in five Kenyan counties. By empowering teachers to assess student progress and achievements effectively, Kurasa helps provide better outcomes for students, teachers, and school management. Where Kurasa is implemented, daily assessments have grown by 253%, with 9 out of 10 teachers recommending Kurasa.

Using evidence to evaluate the effectiveness and guide the implementation of new EdTech models in Bangladesh

Rabia Tanweer and Asad Rahman presented insights from the Bandarban Sandbox, which was carried out in partnership with the education foundation Agami and UNICEF Bangladesh to identify which EdTech interventions are appropriate, feasible, and scalable for learners in remote, low-income communities. 

EdTech Hub’s sandbox methodology tests promising EdTech interventions by leveraging and generating evidence, working in implementation cycles, lean data collection, and iteration in a small cluster of schools within an education system.

The Bandarban Sandbox began by testing five hypotheses on different models of technology integration in classrooms and then focused on three models that showed the most potential. The team developed questions and collected data through teacher and learner surveys, classroom observations, and learner assessments to address initial critical assumptions.

For a rich understanding of how each model worked, the team developed an observation protocol that collected detailed data with guiding questions, answered by descriptive and reflective notes that elicited depth and nuance.

This observation model provided valuable insights into the effectiveness and feasibility of different EdTech models, allowing the team to update and iterate interventions based on evidence. 

For example, the team observed teachers guiding students effectively in 100% of lessons when using TVs or projectors but only in 67% of lessons with tablets. As a result, they concluded that using tablets in class is only appropriate where teachers have high levels of digital literacy and smaller class sizes.

Collecting and presenting accurate and useful evidence: key considerations

At the event, EdTech practitioners and entrepreneurs discussed what is essential when collecting, using, and presenting evidence. Here are some recommendations that emerged from the event.

Gather evidence from different perspectives and stakeholders

In the Bandarban Sandbox, practitioners gathered data from teachers and learners using surveys, interviews, and assessments. 

Nat Dinham, Partnership Lead from the non-profit organisation onebillion and part of the Scaling Personalised Learning Technology in Malawi Sandbox, discussed the usefulness of getting evidence from a proxy product:

“The sandbox methodology helped us uncover that using a proxy product [can be] very helpful”, said Dinham. “We were able to find out a lot about asset management, inventory practices and practical advice on how to store and track assets from someone who hadn’t managed EdTech solutions but had distributed textbooks.” 

Nasrin Ahmadi, previously Monitoring, Evaluation, Accountability, and Learning Officer at Help, also highlighted the importance of gathering evidence not only from teachers and learners but also from external and/or proximal stakeholders to prevent bias.

By collecting data from multiple perspectives and stakeholders, EdTech practitioners gain a more nuanced understanding of the impact of their solutions and make informed decisions to improve their interventions. 

Collect evidence from the beginning

By collecting data from the start of the project, EdTech practitioners and entrepreneurs can design solutions tailored to their stakeholders’ specific needs and clearly understand the context in which they will be implemented. 

Talking about the Bandarban Sandbox, Rabia Tanweer highlighted that it  is about making and testing assumptions. The team always starts the intervention with some assumptions about what it takes for it to work. They then do a series of activities to test those assumptions while knowing what to measure and what to learn from it, and as a result of that, they update their interventions.

Present evidence concisely and effectively

Presenting data concisely and effectively is crucial to get buy-in from stakeholders. Collecting data is not enough — it needs to be presented in a way that is compelling and easy to understand. One group, led by Jennifer Simmons Kaleba, the Head of Communications at EdTech Hub, mentioned that presenting the most succinct takeaways and actionable next steps in a compelling format backed up with data is especially important when liaising with stakeholders such as governments. 

Present evidence from your stakeholder’s perspective 

Kate Radford, EdTech Hub’s Country Lead for Ghana, emphasised that data shouldn’t only be clear and concise — it needs to be presented with your stakeholders’ points of view and needs in mind. For example, for stakeholders like ministries of education, the importance of evidence is to show the impact: how it will result in the necessary changes in literacy and numeracy or if it can be implemented at scale. Kate also highlighted the importance of knowing the drivers and context of stakeholders like governments and ministries, but also caregivers, parents, schools, and teachers.

Rudolph Ampofo, EdTech Hub’s Country Lead for Ghana, also highlighted the importance of adopting your stakeholders’ language and listening to them:

“It’s easy to go and market and talk about your product. It’s very important to listen, so you can tailor your conversation to meet [your stakeholders] where they are.”

As the EdTech sector continues to grow, it is essential to recognise the value of evidence in developing and implementing effective solutions. Events like the EdTech Hub community showcase provide a valuable opportunity for practitioners and entrepreneurs to come together, share insights, learn from each other and collectively build a more robust understanding of the importance of gathering and applying evidence in EdTech interventions.

Do you want to watch the recording of the event? 

Do you want to read more about the Bandarban Sandbox?