Introduction

To positively impact student outcomes across Brazil, Curiós have developed and implemented an intervention that aims to work through improving pedagogical co-ordinator (PCs) and teacher capabilities. To help them develop their evidence base of impact, as well as raise more funding and scale up their program, we developed an evaluation toolkit for them that can be used to generate conclusions that are easy to communicate.

Organization’s role & strength

Curiós is a Brazilian start-up aiming to improve teacher training in the public sector. Every school has a PC, who is responsible for supporting the continued professional development of the teachers in their school. This is Curiós’ point of leverage: by improving the capabilities of the PC, they aim to also increase the impact of teachers on student outcomes. To date, they have worked with over 8,500 PCs across 23 municipalities in Brazil. Their technology based platform is based on a methodology that incorporates the adult learning cycle and encourages individual reflection, collective discussion, practice, refinement and application of the learned strategy into practice. Their values are simplicity, transparency, continuous learning, collective construction, and legacy.

Need summary

Curiós is at an early stage of their journey. They have designed an intervention and successfully implemented it across a range of municipalities in Brazil. However, they are struggling with determining the magnitude of its impact on PCs, as well as for teachers and their ultimate target: improving student outcomes. Crucially, they need a staged approach to building their evidence base evaluating the impact of Curiós, and directions for finding more funding and scaling partners.

Solution summary & next steps

We developed a solution with three parts. The first is a straightforward guide to research design. It outlines the considerations that should be taken into account for a study in their specific context, answers practical questions that might arise during the implementation and even gives a hand-on guide for collecting the data. Overall, we recommend that Curiós partner with an in-country researcher(s) who will be able to support the study’s ethical approval and data analysis.

The second is a set of surveys that would be administered during the research, ranging from concrete tests of understanding of the content to commonly used measures of teacher efficacy and well-being that can be compared with other studies.

Third, we sketch directions they can go to find funding and partners when they are equipped with the results of their evaluation. They can also use this part to find other organizations as examples on how to use research results in communications.

Introduction

Literacy is an essential skill in every language. However, access to books and appropriate fluency interventions is needed to achieve high literacy results in Indonesia.

Bookbot addresses the challenge of teaching children to read proficiently through its innovative learn-to-read app. The app offers a leveled, phonics-based library of books, allowing children to read aloud while receiving natural text-to-speech feedback on their pronunciation and fluency through on-device speech recognition.

With Bookbot, children can monitor their reading skills progress and see how close they are to achieving proficiency. The app also sets goals and offers incentives for reading, rewarding children with a choice of virtual avatars or real-world awards determined by parents and teachers. Designed to be lightweight and efficient, the Bookbot app features a speech recognition and synthesis system that operates smoothly even on low-performance devices without internet access. While the app works offline, it stores data on reading time, fluency, and accuracy in a cloud database, enabling teachers to access and review this information.

Joining LEAP brings Bookbot closer to achieving their vision: that every child has the opportunity to develop their reading, writing and communication skills to live a happy and successful life.

Organization’s role and strength

Literacy is fundamental, not only for our personal and social development but also for our ability to function effectively in society. Bookbot’s vision is that every child should have the opportunity to develop their reading, writing and communication skills to create a happy and successful life. Bookbot’s mission is to deliver tools that use machine learning to advance human learning. The Bookbot app combines speech recognition and a reading program for school children to achieve greater literacy outcomes. Bookbot provides better tools for educators to monitor a child’s reading progress and provide incentives for growth.

Need summary

Bookbot wants to correctly monitor and evaluate its impact and grow its evidence base. Bookbot has articulated the following objectives:

1. Identifying gaps in available data

2. Prioritize additional data collection

3. Providing a structure for data analysis and reporting

4. Develop key evaluation questions

5. Identify key indicators for monitoring

Solution summary and next steps

Following a few discovery sessions with Bookbot and reviewing their organizational objectives, we suggested the following:

Deliverable 1: Strategic problem-solving framework

Deliverable 2: Theoretical evidence framework

Deliverable 3: Documented code

Deliverable 4: Teacher's Guide

Deliverable 5: Recommendations for the future

Regardless of the direction Bookbot is taking in the future, building a solid evidence base provides strong foundations for Bookbot to excel, so most deliverables relate to this. The first deliverable's suggestion came about after learning that Bookbot often faces complex problems involving team members across different cultures.

Introduction

Little Thinking Minds (LTM) is an edtech company with two core digital products for Arabic language learning for native speakers and Arabic language learners. LTM sells its software to 700+ schools in the MENA region and is seeking to grow its user base and scale its impact beyond the 450,000+ students it touches now. Through this LEAP project, LTM sought to:

• Bring more scientific evidence into its product and demonstrate student learning outcomes

• Understand areas of improvement for user retention

Organization’s role & strength

The organization is mission-driven and is attempting to make Arabic language learning engaging through a gamified platform with some traction across several major Arabic-speaking territories. There are ambitious growth plans to enter new regions (like Iraq) and even reach out to countries where Arabic is learned primarily for religious reasons. But the organization understands that this is a competitive market with many similar public and private sector initiatives. It is quite difficult to make learning Arabic engaging and interesting given it is often not central to long-term success in educational curricula across the MENA region. LTM also realizes that to stay on a trajectory of growth with potential for additional funding, it needs to understand how learning outcomes are measured, whether its approach is consistent with the latest in learning (especially reading) science and engineering and whether it is adopting the best approaches to measuring and improving retention, the user interface and gamification.

Needs summary

We can separate the organization’s needs in three areas:

• Scientific evidence / Learning outcomes

• User engagement and retention

• Relaying the integration of more scientific rigor in its messaging

Scientific evidence / Learning outcomes

The LEAP team focused on educating the organization on the “Science of Reading” (SoR) and the potential avenues to design and modify the product based on scientific evidence on how individuals learn to read and how we should teach reading. For instance, LTM products utilize leveled readers, an approach that is centered around assigning primarily those books to children that best match their current reading level. While this sounds like a reasonable approach, the LEAP team advises that LTM understand the limitations of a product utilizing leveled readers based on current research.

User engagement and retention

LTM expressed interest in the LEAP team identifying best practices around retention, user-interface design and gamification to drive engagement. This input was being given in the spirit of some specific feedback as well as general frameworks for experimentation and assessment that LTM can follow in the future.

Solution summary & next steps

Ultimately, we have identified three key recommendations:

1. An overview of SoR and how it is relevant to the context of LTM with a view of explaining to them how evidence-based learning outcomes can be measured and which aspects they should be aware of in thinking about their own learning outcomes.

2. An overview of the game design providing precise feedback on user design, engagement and retention strategies. This includes feedback from people with specialized knowledge in these domains and references to comparable successful products. The purpose of this overview is to provide a framework in which LTM can plan future experiments and approaches to gamification and enhanced user engagement.

3. Review LTM marketing and fundraising materials to integrate messaging around using the SOR or other evidence-based approaches in the curriculum.

Introduction

244 million children are currently out of school worldwide. These children are at high risk of never obtaining functional literacy and numeracy, significantly jeopardizing their futures. AREAi has developed a model to teach out-of-school children in IDP camps in Nigeria functional literacy and numeracy skills, and are planning to scale their model to reach 10,000 children in 2023 and 100,000 children over the next 5 years. This report summarizes the learner journey of children in AREAi's program, outlines alternative approaches to scale-up, and develops a template and comprehensive assessment guide for evaluating the program's impacts in the short- and long-term. The report provides a useful template for other programs aiming to support vulnerable children’s academic development, as well as any organization planning to scale-up their model.

Organization’s role & strength

Established in 2014 and registered in 2017, Aid for Rural Education Access Initiative (AREAi) is a for-purpose nonprofit grassroots initiative that works with and in under-resourced schools and marginalized communities, providing technical and infrastructural support to scale learning outcomes and drive tangible academic achievement for vulnerable children and youth from low-income families. Its mission is to organize, mobilize, and channel human, material, physical, and financial resources towards creating multiple alternative and informal learning opportunities for marginalized populations to attain self-reliance. Through its programmatic interventions focused on foundational skills development, digital equity, girls' education, and school-to-work transitions, the organization has supported over 45,000 beneficiaries from 23 communities in 18 states across Nigeria. AREAi has worked with and received financial support from organizations such as the United Kingdom Foreign Commonwealth and Development Office, the Malala Fund, the Coca-Cola Foundation, Global Changemakers, One Young World, and the Queen’s Commonwealth Trust, among others.

Need summary

Having recently been assessed for scale-readiness by an independent body and placed at Level 2 on the widely-accepted Nesta Standards of Evidence framework, AREAi highlighted, among others, three key objectives:

• Objective 1: Identify key questions to position for scale

• Objective 2: Gain insights to increase impact return on investment

• Objective 3: Strengthen the evidence base for FastTrack

Solution summary & next steps

We were pleased to come alongside AREAi to offer support in two overarching areas, with accompanying deliverables:

• Overview of Program Design

  • Deliverable #1: End-to-end conceptual overview of the learner journey within the context of the program

  • Deliverable #2: Understanding the facilitator role in order to develop a framework for a scale-ready facilitator development program

• Recommendations for Program Evaluation

  • Deliverable #3: A template to conduct a RCT to assess the impacts of the FastTrack program on children’s functional literacy and numeracy

  • Deliverable #4: Recommendations for appropriate assessments to measure learning and other relevant outcomes

Collectively, these deliverables serve as:

A documentation of the core components and ancillary activities of the FastTrack program

A reference, resource, and guide AREAi can draw upon as they develop future iterations of FastTrack

A tool to help AREAi highlight opportunities for engagement in their conversations with potential funders, partners and supporters

We hope that our documentation of the FastTrack model, along with a framework for assessing its impact, spark scholarly interest and innovation / entrepreneurship so that more children, particularly in marginalized communities, have the opportunity to gain functional literacy and numeracy skills and achieve their life goals.

Introduction

VVOB and the Ministry of Education in Rwanda are collaborating to institutionalize continuous professional development (CPD) for school leaders. The effective school leadership professional development programme aims to improve school leadership competences and practices among headteachers and ultimately the quality of teacher performance, and student learning outcomes. School leadership capacity-building programmes in the global North have been found to be effective in boosting student learning gains; but few programmes in sub-Saharan Africa have focussed on enhancing school leadership and its impact on student learning outcomes (Sampat et al., 2020). Through the LEAP project, the ministry and VVOB are aspiring to better understand and measure programmatic impact of school leadership CPD on teachers and students.

Organization’s role & strength

VVOB has a mission of ensuring quality education for all learners. The organisation realises its mission by working with government officials focused on strengthening school leadership and advancing teacher professional development. VVOB invests in research to build the evidence base on what makes professional development effective. It also supports governments to build evidence-aligned programming and take effective innovations to scale.

VVOB’s theory of change starts from a human rights-based approach, ultimately focused on every child’s right to receive a quality education. VVOB works to strengthen the capacity of government institutions that are responsible for the professionalisation of teachers and school leaders. Ultimately, these actors have the greatest impact on the learning performance and the well-being of learners.

Many governments have policy plans in place to continuously improve their education systems in order to achieve SDG4. However, a major implementation gap lies between the existing legal guarantees and policies and the effective realisation of rights and SDG4 targets. VVOB puts these policy plans at the centre of its partnership with the Ministry of Education, and focuses on:

• Reinforcing the capacities of duty bearers, to enable them to guarantee the right to education of the citizens of the nation (rights holders) in a more qualitative and equitable way;

• Promoting equity (the right to non-discrimination) within governments’ education policies and actions.

In Rwanda, the Learning through Assessment and Data (LEAD) multi-year programme for 2022-2026 aims to support SDG4 by ensuring that all Rwandan children can actively participate in quality education and successfully progress through primary education.

Need summary

In Rwanda, the Continuous Professional Development (CPD) programme for school leaders focuses on training and supporting newly assigned school leaders to effectively use data to improve teaching and learning in their schools. The programme is built on a strong evidence base suggesting that with adequate training and support, school leaders can improve the quality of teaching and conditions for learning.

The primary need that is being addressed by the LEAP Fellows is the design of a research methodology to demonstrate the impact of the CPD programme on distributed school leadership and student outcomes. In particular, the research will support VVOB’s partnership with government partners in Rwanda, who seek to understand the return on investment of the CPD programme in terms of student learning outcomes. Investing in research is critical to increasing the credibility of the CPD programme in order to mobilise government financial, human, and policy resources to support program expansion.

Solution summary & next steps

In a first step, the LEAP Fellows conducted a literature search to develop a more fine-grained ToC. The major purpose of this step was to make the different components of the ToC testable (i.e., so that specific hypotheses could be derived). To this end, the LEAP Fellows focused on three main actions of school leaders: providing instructional support for teachers, creating a safe and positive school environment, and collaborating with parents and the community. These actions are hypothesised to improve student outcomes, such as lower dropout rates, better exam grades, and higher pass rates.

In a next step, the LEAP Fellows turned to the planned intervention rollout to suggest a potential impact study design. Because it cannot be guaranteed that the cohort-wise trained schools will be randomly assigned, the LEAP Fellows based their planning on a quasi-experimental design in which schools will be matched between cohorts based on certain criteria.

Finally, the LEAP Fellows provided recommendations on which variables should be assessed at which level (student, teacher, school leader) and how these variables can be used in statistical analyses to test the hypotheses.

Introduction

Pangea’s Project Backpack addresses basic literacy education needs in vulnerable refugee communities using tablet-based semi-autonomous learning. The LEAP project focuses on improving data management, understanding learning effectiveness, and scaling the program. These are all relevant topics to organizations and individuals interested in enhancing education access and quality for underserved populations.

Organization’s role and strength

Project Backpack provides basic literacy education for out-of-school students and students in overcrowded schools. By providing families with tablets, including preloaded educational apps, the project aims to deliver educational content to families in a cost-effective and scalable way. An important aspect of the program is that it empowers parents and peers to participate in their children’s learning, regardless of their prior education.

Summary solution

This project addressed Pangea’s following concerns:

1. How to leverage existing data: As an evidence-driven initiative, Project Backpack has integrated data collection into its implementation. However, the team required assistance making this data more accessible and organized to continue making informed decisions.

2. How to enhance learning effectiveness: Pangea sought a framework to better understand the primary drivers behind semi-autonomous learning in family contexts, aiming to improve their interventions and inform data collection in the future.

3. How to scale up the program: Pangea employs an agile implementation approach for Project Backpack, continuously building on past iterations. They needed guidance on incorporating scaling-up considerations into their current and future implementation plans to expand the program's reach and impact.

Deliverables

1. DATA STRUCTURING TOOL

   A scalable and customizable R script streamlines Pangea's demographic and literacy assessment data management. Addressing common data merging challenges (data quality, data structure, data size, data redundancy, and data privacy) empowers the organization to make informed decisions about interventions and scaling strategies.

2. A LITERATURE REVIEW ON LEARNING IN A SEMI-AUTONOMOUS CONTEXT

   This literature review explores semi-autonomous learning, parental involvement, and peer interaction to enhance learning effectiveness. These concise findings can guide Pangea's next evaluation and inform similar interventions.

3. DATA ANALYSIS PLAN

   The plan includes an R script for initial analysis, guidelines for analyzing existing data and future data collection, and a document on best data practices. This plan aims to assess Project Backpack's efficacy, identify factors influencing its effectiveness based on the previous literature review, and offer recommendations for enhancing evidence-based interventions.

4. SCALE-UP CONSIDERATIONS

   Based on a literature review on scaling up educational interventions, we devised a simplified scale-up framework for Pangea's team, focusing on core scaling elements: effective intervention, implementation, and enabling environment. We adapted VVOB’s Education Scalability Checklist (ESC) to evaluate Project Backpack and develop concrete actions. We also provided three recommendations based on workshop results. To support Pangea and similar teams, we created accessible, user-friendly tools for future use, including workshop guidelines, instructional videos, an action prioritization tool, and a scale-up template.

Introduction

Schools in informal settlements in Kenya provide access to crucial education for many young students, yet rates of finishing primary school are low. CFK Africa is working to improve attendance via a variety of methods, and this LEAP project was designed to provide guidance on how to implement, measure, and eventually scale specific interventions to improve school attendance in informal settlements in Kenya.

Organization’s role & strength

In Kenya, millions of people live in informal settlements. In these communities, residents lack access to quality health care, proper nutrition, and adequate sanitation, leaving them susceptible to preventable diseases. Limited access to quality education and economic advancement opportunities, high levels of unemployment, and gender inequity further impede individual and community progress. Though residents are talented, resilient, and entrepreneurial, many lack the opportunities needed to break cycles of poverty. CFK Africa is working to change that narrative.

CFK Africa’s Best Schools Initiative (BSI) works with students ages 5 to 12 and collects data from 64 primary schools in Kibera. This initiative targets equitable and sustainable education development at multiple levels; building capacity at the school level, assisting school leaders in implementing transformative policies, and helping guide the greater educational community in leading similar research.

Need summary

Schools in informal settlements are critical for providing accessible education for many students, but it can be difficult to keep attendance rates high. CFK Africa, through their Best Schools Initiative, worked closely with stakeholders to identify 12 factors (for example, providing school lunches, teacher training) that could potentially improve school attendance in informal settlements. They are now implementing these practices in partnership with schools, but had questions about data collection and levels of evidence required to evaluate the practices. The focus of this LEAP project was therefore to provide advice and strategies for collecting data to evaluate these practices and providing a framework in which to do so.

Solution summary & next steps

We worked to provide value to CFK Africa and for the wider field through four deliverables.

First, we used CFK Africa’s existing data (collected from schools in 2020, prior to any interventions) to glean insights to guide current and future data collection, and to provide preliminary evidence to guide future interventions.

Second, we created an overview of different research design methods to measure intervention effectiveness and infer causality, and we discuss the advantages and disadvantages of each in the context of CFK Africa’s Best Schools Initiative.

Third, we provide advice on the best data collection platform and methods to measure school attendance and other relevant data.

Finally, we provide advice on how to pursue potential funding and partnership avenues to sustain effective programs, as well as potential venues for sharing this important work with others.

Introduction

Thate Pan Hub (TPH), a Myanmar-based social enterprise building computer science learning opportunities through a gamified platform, is on the outer curve of innovation. They are figuring out how to create easily accessible learning opportunities in an unstable political landscape, in a society shaped by decades of armed conflict, and in an educational system not fully supported by an ecosystem of institutions. These contextual factors place TPH in a humanitarian aid and international development category, yet, at the same time, they are able to draw upon leading educational approaches that we all hope might leapfrog them into a new category of greater impact. If TPH is successful in their work, they would be a model for how to deliver culturally-relevant educational opportunities for a difficult-to-reach group of learners, and their doing so would have great implications for delivering education in emergencies.

To delve deeper into how their work could positively impact students, TPH responded to a call for proposals for organizations looking to scale their operations in education and expand the evidence base of their impact. TPH was one of several organizations selected for the opportunity, which came with 12 weeks of support from a team of consultants working with them to address a challenge they face. The project is a collaboration between The Jacobs Foundation, MIT Solve, LEAP Fellows, and the organizations themselves.

TPH was paired with a team of four LEAP Fellows who have considerable experience and expertise in social entrepreneurship (having founded and/or operate social enterprises themselves), and research (many of whom hold academic positions at universities).

Organization’s role & strength

Coming into the project, TPH proposed working with the fellows to expand the menu of features for their computer-science-focused platform. Their proposal stood out because it signaled that they were motivated to build, get the app out there, and get it into the hands of Burmese students for the children’s use and benefit.

Need summary

Settling into the project more, we realized that helping them build out product features would be useful, but, we also quickly realized that, given the short timeframe, it would be many times more impactful to spend the time helping them build the skills to approach product, organizational, and design challenges they will for certain face as they grow. This approach will set them up with a more comprehensive capability as a team long after this engagement ends.

Solution summary & next steps

The revised project was ultimately a program of workshops with a strong focus on experimentation. The experimental parameters were to identify a problem or challenge or pose a question, and then gather the research and evidence to develop a potential solution.

Over the course of these 12 weeks, we conducted six workshops in keeping with this theme. The focus of each is below:

• In Pedagogy, we focused on the relationship between user testing and the product’s theory of change

• In Design, we focused on activity-based research methods

• In Technology, we focused on high scalability with under-resourced orgs

• In Business Model, we focused on rapid iteration to generate a bank of options

• In Product Roadmap, we focused on user testing to guide development and

• In Strategy, beyond the recommendations we made, we provided TPH with the resources for them to periodically check-in with themselves and draw upon various interviewing and research techniques, which will ultimately help them arrive at a more well-informed decision

The overarching question that TPH is trying to answer is: “How might we increase CS learning outcomes for children and youth in a post-conflict, emergency, humanitarian, and development context?” Undoubtedly, TPH will face many more product, organization, and expansion decisions in the future, and this framework of investigating and exploring is a tried and tested method to gather field-based information to better inform the decision-making process.

Over the course of this report, the various activities conducted with TPH are summarized and serve as a tangible example on how TPH can use experimentation methods to support organizational or product decisions in the future.

Introduction

LEAP Fellows have produced a case study (downloadable below) documenting how Amal Alliance (hereafter referred to as Amal) has scaled its SEL product, Colors of Kindness, both in formal and informal education settings. The case study focused on the following key areas:

• How did Amal build its evidence base?

• How did Amal influence stakeholders in both formal and informal education settings

• Scaling Set Backs & Opportunities

• Next Phase of Growth

The LEAP Fellows conducted regular interviews with Amal as well as an agile digital Q&A.

The case study is an easy-to-digest product, which has three key audiences in mind:

• Founding Teams

• Governments

• Funders

The product is intended to serve as: 1) public good that Amal will publicize and disseminate to other NGOs and stakeholders, and 2) a tool that Amal can use to continue its scaling journey.

Organization’s role & strength

Founded in 2017, Amal Alliance (Amal) is a nonprofit, US-based Non-Governmental Organization (NGO) dedicated to empowering displaced and disenfranchised children through education and social development programs. They provide 4 pillars of support, 1) Social Emotional Learning, 2) Early Childhood Development, 3) Psychosocial Support, 4) Peace Education, to children around the globe in both refugee camps and formal school settings.

Amal Alliance’s nimble and agile nature allowed for an innovative response to both the education and growing mental health crises during and post COVID-19. When the pandemic hit, not only were already vulnerable children at further risk, but all children were stripped of their educational pursuits and ability to play with their peers. Like the rest of the world, Amal was forced to pivot from their traditional in person classes and explore how to best leverage technology. They quickly repackaged their social and emotional expertise in a dynamic and innovative way that garnered rapid traction by making SEL accessible to communities that were underserved. Across its four SEL programs – Colors of Kindness (a blended learning program), Superhero Academy (a home-based learning program), and Rainbow of Education (a teacher training program with accompanying curriculum), and Mini Rainbow (curriculum for 3-6 year olds) – Amal has reached over 276,000 children across 43 schools and other educational settings in 5 countries, 3 continents, and across 4 geographical locations in Bangladesh, Greece, Lebanon, Turkey, and Uganda.

As a volunteer-led core team, they have skillfully perfected the art of leveraging partnerships, networks, and key stakeholders to further the collective mission of all, whilst building an evidence base for SEL within education in emergencies.

Need summary

Amal sought to create a comprehensive roadmap on how they integrated social and emotional learning (SEL) into both formal and non-formal education systems in Greece, Bangladesh, and Uganda.

The roadmap would capture the best practices and strategies that were used to build institutional capacity among various educational stakeholders, and how important strategic partnerships to mobilize collective efforts, and evaluate the effectiveness of interventions. Given the lack of research in this area, it would serve as an invaluable public good, with the potential to exponentially increase impact by sharing the learnings across both the education and humanitarian sectors.

Amal’s team lacked expertise in marketing, communications, and research, which required the LEAP global cohorts knowledge to create this communication tool. Ultimately, Amal hopes this tool will provide valuable learning opportunities for other Founding Teams, Governments, and Funders to understand the model and use it as a case study on how to move towards holistic whole child development.

Solution summary & next steps

The case study provides a set of best practices and practical insights for other early-stage ventures seeking to build their evidence base and influence stakeholders.

Introduction

Street Child runs Community Based Education Programs in remote and crisis-affected regions of Afghanistan with a strong focus on improving foundation literacy skills among the Out-of-School-Children. They recruit and train teachers, as well as coaches to support them. A 3rd party Early Grade Reading Assessment showed that the “Street Child programme is doing very well in ensuring that reading skills of students are improving compared to other reading programmes in Afghanistan” (End Line Evaluation Report of Multi-Year Resilience Programme-Uruzgan)

However, Street Child’s evaluations have also shown that there is low fidelity to prescribed teaching practices especially among novice teachers in hard-to-reach districts of provinces like Zabul; they are determined to address this.

Through the LEAP Project, Street Child and the LEAP team are seeking to understand the challenges community-based education teachers in Afghanistan face in translating professional development into practice. Together our goal is to identify strategies to better support Street Child’s teachers to apply what they have learned and to translate these practices with fidelity and quality into improved student learning.

Organization’s role & strength

Street Child ensures that children are safe, in school and learning even, and especially, in low resource environments and emergencies. The scale and scope of challenges affecting children in emergencies and low resource environments, especially in the era of the COVID-19 crisis necessitates agile and ambitious action. Street Child is committed to confronting these challenges through purposeful programmes, and pledges to make a meaningful, material contribution to increase safety, access to schooling and learning for a million children during 2021-24 and millions more through its powerful partner network. By targeting the most marginalized, tailoring and integrating its response for them, accelerating local level action and by assessing impact, Street Child is on an illuminated path to redeem its pledge.

Street Child Afghanistan works across 15 provinces in Afghanistan providing humanitarian relief and response. In 2022, Street Child made up 13% of all community-based education (CBE) centers across the country. In 2022-23, more than 10% of learners in community-based classes in Afghanistan were attending Street Child run classes.

Street Child uses the Early Grade Reading (EGR) curriculum developed by the Ministry of Education (MoE) of Afghanistan in collaboration with the USAID funded Afghan Children Read initiative (ACR). An important aspect of this work is improving teachers’ instructional practices at scale to improve student learning.

Need summary

It is widely recognized that teachers are the strongest school-level predictor of student learning (Scwhille et al., 2007). The critical role that teachers play in delivering quality education is amplified in humanitarian crises, and yet teachers in these contexts face some of the most difficult teaching conditions. Moreover, these teachers often have little to no teaching experience, work in under-resourced environments, and receive professional development support that is uneven, inadequate and fails to meet their needs (INEE, 2015).

Several of these challenges are relevant to Street Child’s EGR intervention in Afghanistan: teachers often have no teaching experience; resources are limited; and, while they are supported by teacher coaches, coaches are often new to teaching themselves and struggle to model prescribed practices and support teachers to improve.

Program evaluations of Street Child’s EGR intervention in three provinces highlighted a common issue: low fidelity to prescribed teaching practices compounded by weak evidence of the impact of coaching.

Given the layered nature of Street Child’s intervention (working with teachers, teacher coaches and coordinators at the provincial level) identifying at which levels of the program leakages and barriers to implementation were occurring required investigation.

The LEAP project team’s approach was to work with Street Child to review the wider evidence base and gather insights from Key Informants (KI) involved in the program with the goal of locating where the main barriers to fidelity might exist and identifying strategies that will have the greatest impact on improving teachers’ uptake of the prescribed teaching practices.

Solution summary & next steps

We anchored the project in a simple problem statement agreed with Street Child:

There is low-fidelity to prescribed teaching practices among novice teachers in the targeted hard to reach districts.

And we set the following simple objectives to guide us:

To increase:

• Fidelity to prescribed teacher practices

• And effectiveness of coaching services

By:

Identifying the root causes of low fidelity and ineffective coaching.

Developing a set of recommendations drawn from relevant research and best practice.

Creating a feasible implementation and evaluation roadmap for the Street Child team.

Our team used a strategic problem solving framework to structure our work. We adapted the framework developed by the Yale Global Health Leadership Institute (GHLI) which offers a “rigorous and systematic approach to finding practical, focused solutions to complex problems.” (see Annex 1)

We organized our process according to four sequential phases:

1. Define the problem and Gather Evidence

2. Conduct a root cause analysis

3. Generate and Compare Strategic Options

4. Create an Implementation Framework

Each of these phases produced their own set of deliverables and outcomes that informed the phases that followed.

During “Phase 1: Define the Problem” our team conducted a literature review to synthesize the available evidence. We developed a survey tool to gather insights from Key Informants involved in the program and for Street Child to use in any future data gathering efforts.

In “Phase 2: Conduct a Root Cause Analysis,” our team used the evidence from our literature review to develop different categories that are known to influence teacher fidelity and used this to map the different factors that could be contributing to the problem in a root cause analysis.

In “Phase 3: Generate and Compare Strategic Options” our team developed a table of strategic options grounded in evidence. We then designed a decision making matrix to perform a side-by-side comparison of the alternatives with Street Child, using evaluative criteria to select the best option - restructuring the coaches’ training to increase coaching efficacy and provide improved ongoing support to teachers.

Lastly, in “Phase 4: Select the Best Intervention,” our team developed an implementation framework that outlined key steps and considerations towards increasing coaches’ efficacy through improving their practical training.

The result is a research-based roadmap for the Street Child team on improving teacher fidelity that highlights key evidence and strategies on teacher professional development and coaching in international development and education in emergencies (EiE) and contains tools that other organizations can draw from when embarking on similar evidence gathering and strategic problem solving efforts.

Introduction

Teach the World Foundation (TTWF) is a non-profit headquartered in Pakistan, that designs, develops and operates MicroSchool, In-School and SmartPhone programmes to enable young people to learn basic and functional literacies. Aside from broader corollary societal benefits of basic literacies, the programme enables students in Pakistan to gain access to the formal education system and consequently embark on their journey of lifelong learning and all the accompanying benefits that affords. TTWF has established evidence, through existing studies, that students achieve intended learning gains within the TTWF programmes and through social proof that students are able to transfer their basic literacies in their communities to functional literacies.

The MicroSchool programme is a ground-breaking initiative that enables the affordable setup and implementation of a school for 100 students led by a facilitator in a community within a week. The programme occupies existing available buildings. Learning by students is self directed and delivered through educational apps on tablets that are able to function in online and offline environments. Data about learning is available through an evolving set of app dashboards and is also collected through periodic EGRA, EGMA and EGRA-Urdu testing.

TTWF are now embarking on a first step in scaling their models through an initial partnership with the Sindh Government to establish 100 MicroSchools and 25 InSchool programmes against a schedule of operational and educational KPIs.

As TTWF progresses into the execution of the Sindh project, with an eye on the future, it now seeks to establish longer term KPIs that can demonstrate meaningful and robustly evidenced outcomes for learners and communities as well as meaningful, confidently defined and compelling operational and educational outcomes for future partners and stakeholders that will be key to the long term growth and impact of its initiatives.

TTWF has been paired with a team of four LEAP Fellows with experience in social entrepreneurship (2 Fellows) and research (2 Fellows) and whose experience in data analysis, scaling educational programmes and data collection will support the aims of TTWF in this project.

Organisation’s role & strength

With an existing base of evidence and research, TTWF came into the 2023 LEAP programme with an eye on scaling the breadth of impact that can be measured in their programmes. Organisationally TTWF is well organised with dedicated and experienced team members focused upon the evaluative, educational, technological, operational, business development and strategic aspects of programme implementation, measurement and growth. There is an existing culture of seeking to understand the efficacy and evidence of programmes that is pervasive across the organisation and drives decision making. TTWF has begun to consider the operational realities as well as the practicalities of a broader spectrum of impact measurement and is seeking to implement an enhanced technology architecture to support this.

Need summary

The first four weeks of the project with TTWF began with Fellows examining existing data collection practices, available data and its corresponding analysis from across prior programmes and assessment exercises that had been previously conducted. Fellows met with TTWF team members from strategy, education, evaluation, technology, operations and business development teams. This gave a clearer understanding of the objectives and vision behind an expanded impact measurement, considering the goal through the lens of different groups within TTWF who are tasked with meeting different immediate and longer term programme requirements (e.g. operational considerations for communities accepting a Microschool implementation), as well as existing stakeholder KPIs (e.g. Sindh Government KPIs for operational outcomes of the Microschool and In-School interventions).

It became apparent through this discovery period that there was a need to establish clear, consistent and robust benchmarks for the analysis of existing learning progress data within the programmes. TTWF uses EGRA, EGRA-Urdu and EGMA assessments, which whilst being well established across literacy development, do not come with a consistently comparable group of international benchmarks from peer initiatives, or that apply to the learning of Urdu. Therefore, it would be important to understand how to establish these during the LEAP project and to set scalable practices and benchmarks for the collection and analysis of future learning progress assessment going forward.

Many of the broader spectrum of impacts that are often cited from educational programmes quickly convert into economic measurements. However, as we developed an understanding of the specificities of the stakeholders that are and will be instrumental in the sustainable scaling of the TTWF programme in the long term, it became apparent that the societal impact must relate to expressed needs and therefore those perceived as compelling outcomes by learners, communities, governments, donors and future non-profit partners. This also serves the opportunity to focus the scope of outcomes to be considered, their potential for measurement and required interdependencies thereof.

Solution summary & next steps

At the conclusion of the discovery phase of the project the deliverables were concluded as follows:

1. How can TTWF scale its research roadmap to capture both specific and broad impacts in line with identified needs of current and future stakeholders, including learner competencies, community impact, and overall societal benefit? How and when can these be measured in a way that is scalable and alongside product development? How can other comparable programmes be used to better understand potential benefits of TTWF?

2. In line with #1, how can existing data on learning outcomes be used to make a compelling case for learning gains associated with TTWF programmes? How can other existing data (not learning outcomes) be used to help persuade communities and governments of the value of TTWF programmes?

3. What further data collection possibilities (concerning the learner and beyond) will be required to help respond and be compelling to the needs of the different stakeholders?

The outputs of these deliverables will seek to place TTWF in a position for its teams to proceed in existing programmes with a consistent and robust means of assessing and analysing learner outcomes as well as establishing a data collection and evidentiary framework for sustainably measuring and demonstrating the broader impacts of the TTWF programmes on society at large.

The benchmarking of learner outcome assessments, particularly within the context of an organisation assessing progress through EGRA and EGMA, will serve as a valuable reference for other programmes seeking to establish similar consistency and robustness in evaluating learning progress.

The broader societal impact framework begins from a consideration of societal impacts that can have relevance to a cross section of educational initiatives. The process through which its measurements can be focused to be implementable and relevant to an individual organisation’s programmes and stakeholder considerations is something that other organisations can implement as a transferable approach to demonstrating a broader spectrum of societal impact themselves.

Introduction

Literacy is an essential skill in every language. GLOT organization, through its Literacy+ program, seeks to assess and improve the reading, writing and math skills of Colombian school-age children and adolescents in need of additional learning support using targeted instruction and play-based learning methodologies. It utilizes a play-based learning kit and teaching material combined with teacher training and ongoing support for the implementation of targeted instruction with which these skills are effectively improved.

Organization’s role & strength

Mission:

We have been fortunate enough to have learned many languages! We can also recognize its transformative effect on a human and social level. With this as our passion, we provide educational opportunities so that the power of language is accessible to EVERYONE.

Vision:

As a cultural transfer and social impact network, our vision for 2023 is to connect different actors and reduce barriers related to languages by increasing educational opportunities through:

• Empowering every child to be the best version of themselves by mastering literacy & numeracy through play-based and targeted instruction pedagogies.

• Empowering multilingual professionals from the global south by providing equal opportunities, mentoring, and resources.

Core values:

• Love

• Multilingualism

• Respect

• Gratitude

• Sustainability

Need summary

GLOT wants to correctly monitor and evaluate its impact, grow its evidence base and broaden its sustainable funding streams.

GLOT has articulated the following objectives:

1. Become more sustainable

2. Include a focus on displaced children

4. Grow the program intervention

Solution summary & next steps

Following a few discovery sessions with GLOT and reviewing their organizational objectives, we suggested the following:

Deliverable 1: Impact & Social Lean Canvas

Deliverable 2: Literature review

Deliverable 3: Mapping the learner journey

Deliverable 4: Mapping the teacher journey

Deliverable 5: Recommendations for the future

Irrespective of the path GLOT may choose to follow in the future, establishing a robustfoundation of evidence is crucial to enable GLOT to thrive. As such, a significant portion of the deliverables are oriented toward this goal.

Introduction

Dignitas partners with schools across Kenya to provide evidence-based training and coaching that strengthens school leadership and improves instruction. The LEAP Project aimed to provide key resources to underpin Dignitas’ current work focused on inclusion of marginalised, neurodiverse children in Kenya’s schools. Evidence-based resources included a literature review, learning briefs and a competency framework. Together we hope these resources support Dignitas in their mission to transform opportunities for the next generation.

Organization’s role & strength

Dignitas is a leading education organisation whose mission is to equip and empower school leaders and teachers who will transform opportunities for the next generation. Dignitas partners with schools to strengthen leadership and improve instructional quality, in order to create an environment where all children can fulfill their potential.

Dignitas is founded on the belief that School Leaders and Teachers are everyday superheroes with the power to transform children's futures when supported with the right training, support, and community. The following values guide us:

• We are passionately purpose driven

• We co-create solutions with schools, communities, and system actors

• We find joy in our work, our team, and our communities

• We found partnerships in trust and transformative impact

• We are always listening and learning

Need summary

As Dignitas seeks to support schools to advance inclusion of neurodiverse learners, the organisation identified four key questions for the fellow team to address through four key deliverables. The questions focused on:

• the current status of neurodiverse children in underresourced settings in Kenya;

• effective instructional strategies for the inclusion and support of all learners;

• the mindsets, competencies, and practises school leaders and teachers need to embrace learning variability; and

• how teachers, school leaders, and school leader supervisors can better collaborate to promote integration of neurodivergent children.

The fellows sought to answer these questions through a literature review, learning briefs, and a competency framework, each designed to provide actionable information to the Dignitas team and school leaders working with marginalised neurodiverse children in under-resourced settings in Kenya.

Solution summary & next steps

Across a 12 week sprint the fellow team, in close collaboration with Dignitas, developed four key deliverables. The first was an extensive literature review that covered the four key outstanding questions highlighted by Dignitas at the beginning. This literature review was supplemented by two concise, actionable Learning Briefs. The first Learning Brief focused on how teachers, school leaders, and school leader supervisors could better collaborate to promote integration of neurodivergent children. The second Learning Brief covered the effective instructional strategies for the inclusion and support of all learners, and specifically neurodivergent children, in under-resourced settings. Finally we developed a single slide competency framework to provide a clear set of four key competencies for teachers and school leaders to enact inclusive educational practices. Together, these deliverables will augment Dignitas’s existing evidence-base and provide clear and up-to-date evidence that can support the organisation as they provide training and coaching to schools focused on inclusion of neurodiverse learners. Dignitas also plans to share some of these deliverables more widely to become a public good for the field at-large.

Introduction: Aflatot Program

Aflatot is an early childhood education program for sustainable development, developed by Aflatoun International specifically targeting preschool children. The program aims to develop age-relevant social-emotional skills and financial awareness and features training for facilitators to deliver the program through an active learning methodology.

The Aflatot curriculum is designed for children aged three to six years old and aims to lay the foundation for social and financial literacy before they reach primary school. By balancing social learning and financial concepts, Aflatot empowers children to believe in themselves, understand their rights and responsibilities, practice saving and spending, and even start their own enterprises.

The program includes a curriculum with detailed lesson plans or activities designed for educational settings such as schools, nurseries, and community centers. The program has a duration of 45 hours. In addition to the curriculum, Aflatot provides a family toolkit that offers sessions facilitated by educators, engaging parents with their children to learn together about social and financial content. The toolkit also suggests activities that can be done at home.

Organization’s role & strength

Aflatoun International has 17 years of experience in life skills and financial education programming worldwide. The organization’s mission is to ensure access to quality, inclusive, child-centered social and financial education for all children and young people, especially the most vulnerable. Aflatoun’s overall goal is to socially and economically empower and provide children and young people with the skills, knowledge, and resources necessary to participate and thrive in their communities. Aflatoun, as a social franchise, currently works with and provides technical expertise to a partner network of 345 organisations (including NGOs, CSOs, and governments) that implement social and financial education programmes in more than 100 countries across different regions.

Need summary

Aflatoun was seeking to understand the following in relation to the Aflatot program:

• Need 1: What is the short-term and long-term impact of the Aflatot programme on the socio-emotional development, financial literacy, and behaviour of young children, particularly in terms of their self-regulation, delayed gratification, decision-making skills, and understanding of money? How can we further improve the programme’s content and delivery?

• Need 2: What is the current state of social and financial skills development in early childhood education? What is the impact of social and financial skills development programs targeted at preschool-age children in early childhood education, and what are the research gaps in understanding their effectiveness?

Toward addressing the Project Host’s needs, we developed two deliverables:

• Deliverable 1: A state-of-the-art literature review and report specifically addressing the questions posed.

• Deliverable 2: A research feasibility report to evaluate the implementation methodologies, necessary resources, and design strategies for conducting an effective Randomized Controlled Trial (RCT) on preschool-age children's social and financial skills development.

Introduction

Blue Butterfly is a nonprofit organization devoted to inspiring a generation of learners through the power of engaging educational media. In partnership with the Model School Network, Blue Butterfly is equipping teachers in Haiti to deliver “Apwòch Lekòl Vivan” (ALV), a learner-centered pedagogy to meaningfully engage children in learning. By developing Eksploratoryòm, a story-based STEM education program designed specifically for the Haitian context, teachers are able to engage students in science and improve student learning outcomes.

The Blue Butterfly LEAP project focused on A) strengthening Blue Butterfly’s theory of change through a literature review defining student engagement and its links to learning outcomes, and B) identifying tools to measure student engagement, including considerations for appropriate use and context.

Organization’s role & strength

Blue Butterfly’s Eksploratoryòm program is addressing the root causes of poor educational attainment in Haiti by creating innovative, culturally relevant tools, resources, and capacity to transform teaching and learning, ensuring that Haitian children receive the holistic education necessary to thrive and succeed. We are bringing evidence-based educational innovations to Haiti and customizing them to match the realities of life in Haitian communities.

Our core values are:

• Cultural Relevance—Children deserve educational materials that feature settings and characters they can relate to.

• Local Production—We are building the capacity of our team and partners in Haiti to produce high-quality educational materials.

• Educationally Rigorous—Our Eksploratoryòm program meets educational objectives set by the Haitian Ministry of Education

• Entertaining—We develop fun stories with engaging characters that appeal to children because learning can be fun!

• Deep Local Connections—Blue Butterfly’s network of community partners help us test and implement our programs in Haitian schools and homes.

• Home Language—Research shows that children learn best in the language they speak at home. Our Eksploratoryòm program is exclusively in Haitian Creole, the native language of most Haitians.

Need summary

Blue Butterfly’s theory of change states that by meaningfully engaging students in their learning, the Eksploratoryòm program leads to improved student learning outcomes. A pilot study of Blue Butterfly including 980 students indeed found that children in Eksploratoryòm classrooms

showed greater learning gains along the four curricular themes than those in the comparison group. However, given the expense and difficulty in measuring student learning directly in Haiti, Blue Butterfly was interested in exploring a rationale for measuring student engagement. They wanted to know if there is strong evidence in the literature linking student engagement and learning outcome, justifying measuring student engagement as a proxy for learning. They also wanted to know what measurement tools were available to evaluate student learning, and under what circumstances those measures might be appropriate for Blue Butterfly’s use.

Recommendation summary & next steps

The LEAP project found that the literature does support a strong link between student engagement and student learning outcomes. The Fellows completed a literature review defining student engagement and its components, exploring the factors contributing to student engagement, and summarizing the evidence that high levels of student engagement are predictors of high levels of student learning. The literature review also explored the components of Blue Butterfly’s Eksploratoryòm program and their relationship to student engagement, including the program’s focus on cultural relevance, school context, joy, teaching quality, and quality curricular content and material. Blue Butterffly will use the literature review to support its rationale for evaluating its impact on student engagement, and for others hoping to advocate for the importance of student engagement in learning.

The second LEAP project deliverable is an analysis of the tools available for measuring student engagement. These tools include the PLAY Toolkit, LENA, TeachFX, and the Stallings Classroom Snapshot (a questionnaire measuring teacher engagement was also identified). The LEAP Fellows created a table describing the training, administration and cost requirements for each tool, as well as assessing its applicability to the Haitian context. In summary, the PLAY Toolkit, LENA, and TeachFX had potential for generating rich data-based insights on student engagement, although they varied in their respective financial and administrative cost required. This tool analysis will serve as a helpful guide to Blue Butterfly and others in deciding which tool to use to evaluate student engagement under a given set of circumstances.

The third LEAP project deliverable is a set of recommendations for further study. Drawing from the LEAP Research Fellows’ areas of expertise, these recommendations could further bolster Blue Butterfly’s theory of change.

Although research shows that early literacy and numeracy are foundational to later achievement and employment, in South Africa, 82% of children in Grade 4 cannot read for meaning (Spaull, 2023) and 63% of learners in Grade 5 have no basic mathematical knowledge (HSRC, 2020). These educational deficits compound as learners move through the education system and contribute to South Africa’s high youth unemployment rates (Click Learning, 2024; O'Neill, 2024).

Given the complexity and magnitude of the learning crisis, traditional change efforts have often fallen short. Educational technology (edtech) has emerged as a promising solution to enhance learning outcomes and promote equity at scale. However, there is limited evidence regarding which programs are truly effective, for whom, and under what conditions. Although some intriguing trends highlight emerging best practices—like supplemental or after-school interventions and adaptive or personalized technologies—there are also common pitfalls. These include a mismatch between products and learners' needs, particularly concerning language, as well as various contextual factors that can impede the effectiveness and sustainability of edtech interventions.

These considerations are particularly vital in South Africa, where schools serve multilingual learners and grapple with infrastructure and human resource constraints. In this complex landscape, organizations like Click Learning, a South African NGO, play a crucial role in delivering locally led and contextually relevant solutions.

Click Learning's mission is to equip learners with the foundational literacy, numeracy, and digital skills necessary to build sustainable livelihoods in the future. Employing a holistic approach, Click Learning offers underserved schools access to top-notch digital learning programs, along with vital infrastructure (such as hardware, connectivity, and backup power) and essential human resources (including lab facilitators). By facilitating the integration of edtech in low-resourced schools, Click Learning is paving the way for thousands of children to succeed in school and beyond.

In the last ten years, Click Learning has collaborated with more than 340 schools, benefiting 230,000 literacy learners and 50,000 numeracy learners. Inspired by findings from a recent study indicating a significant link between enhanced learning outcomes and learners' cumulative time spent on Click Learning programs (Firdale, 2022, p.13), Click Learning partnered with a team of LEAP Fellows to investigate the most effective ways to structure learners' usage for optimal gains. This investigation aimed to determine the minimal or optimal time learners need to meet global proficiency standards in literacy and numeracy by Grade 3. Such insights would enable Click Learning to estimate the ideal balance between the number of sessions and the time spent per session and ensure that the time allocated for Click Learning results in high-quality learning.

During a three-month sprint, the team conducted a literature review to understand the evidence base related to edtech dosage and learning outcomes. After reviewing several studies, including multiple meta-analyses, the evidence remains limited, particularly within the South African context, regarding the impact of dosage, intensity, or duration of use on learning outcomes. This underscores the importance of studies like the present one.

The team also explored the data available for two of Click Learning’s programs, Reading Eggs and Reading Eggspress, examining associations between students’ use of the platform and their gains in reading comprehension from one year to the next. Below is a summary of our findings:

• The study uncovered significant variability in weekly logins and platform usage among individual students, between students, and across schools, highlighting inconsistent patterns likely influenced by both internal (e.g., student preferences) and external (e.g., school policies) factors. Such variation can be leveraged in analyses of the links between time spent and performance gains, but variation driven by students indicates that confounding variables related to both time spent and performance may bias such analyses.

• In non-regression-adjusted results, platform usage variables from Reading Eggs were negatively associated with reading comprehension scores from Click Learnings’s equiz, whereas those from Reading Eggspress showed positive associations. These perplexing results may stem from outliers or students accessing the platform beyond their grade level. For instance, our analysis focused on grades three through five, although Reading Eggspress targets grades six and seven. This suggests that students using Reading Eggspress in lower grades may be unusually proficient, or Reading Eggs might be too elementary for them. Further investigation is necessary for clarification.

• There was substantial agreement between the data mined Reading Eggs time-on-platform variable and Click’s previously calculated platform time variable. However, this agreement was not found for the time-on-platform variable in Reading Eggspress. This suggests that an aggregated variable, such as the previously calculated one, may not capture the nuances in the time spent on specific programs within Click Learning.

• In the regression-adjusted results, time on either Reading Eggs or Reading Eggspress was associated positively with score on the external reading comprehension test. For both programs, when students spent a standard deviation more time on the program each week (approximately 10 minutes per week), they gained one-fifth of a standard deviation (0.20) in reading comprehension score. This is a meaningful effect size in the context of reading instruction within South Africa.

• From these analyses, it appears that for this sample, longer sessions are associated with greater gains in reading comprehension. For both programs, the time per login variable was a positive statistically significant predictor of reading comprehension score, with a beta of 0.07 for Reading Eggs and 0.11 for Reading Eggspress. Examining this balance of number of logins by time spent through an interaction term, the interaction of time by logins was only statistically significant for the Reading Eggspress model. In these data, it appears that the association between time spent on Reading Eggspress and reading comprehension score is stronger for those students who have fewer logins. This indicates that students should be given more time per session to engage with Reading Eggspress. These results should only be interpreted as preliminary given that there were a number of extremely short logins (less than two minutes) that may be skewing results.

Based on insights gained during this sprint, and informed by the literature reviews, the report offers customized recommendations related to:

• the data collection and analysis of Reading Eggs and Reading Eggspress;

• further exploration of the connection between dosage and learning outcomes;

• the broader language and learning context of the Click Learning program;

• and enhanced program implementation and evaluation.

A great benefit of the LEAP program is that it allows host organizations and several experts in their respective fields to rally around a collective question: how can the host organization supercharge their path to have rigorous evidence of the effects of their program?

Over the course of a team sprint this Spring, we set out to address that question for the Seenaryo Playkit - a teacher training delivered alongside an app-based learning tool, that provides easy access to hundreds of play-based learning activities that are designed to build social and emotional learning (SEL) and life skills for children ages 3-8 in Jordan and Lebanon. The Playkit is a program of Seenaryo who operate in Jordan and Lebanon and to date has reached over 400 schools, serving over 89,000 children.

Seenaryo has been operating in the field in Lebanon since 2015 and Jordan since 2018 and is one of few organizations championing play-based learning as an essential component of early childhood instruction in the region.

As we began to answer this collective question of “How Might We Supercharge the Path to Rigorous Evidence of the Program Effects,” several sub-questions emerged:

• EVALUATING IMPACT: How does Seenaryo ready themselves as an organization for a future experimental study of the effects of the Playkit?

• CHILD EFFECTS: What trusted/tested/rigorous methods might Seenaryo use to observe the effects of the Playkit on children in the classroom?

• GREAT DATA: What are some of the best practices of capturing and analyzing Playkit data and how does Seenaryo build that capability?

• APP DATA: How can Seenaryo get the most information from the app and tech-based environment (surveys) while optimizing automation?

As we explored these many questions in the sprint, a focus to our work emerged when Seenaryo shared what we began to refer to as their North Star Metrics -- the metrics that they theorize would demonstrate positive intended effects on their students as they were designed.

The impetus to build a stronger evidence base is guided by Seenaryo’s interest in building a better and more effective product, demonstrating that the product is having intended positive effects on children’s learning and outcomes, and build a case to drive greater investment in Seenaryo so that they may scale within Jordan and Lebanon, and set the stage for additional regional scaling.

Introduction

The global community is at risk of failing a generation of young people in the Global South who may never become the doctors, leaders, human rights activists, or teachers they might have been. Worldwide, it is estimated that 70% of children are not learning to read by the age of 10. An estimated 82 million primary-school-aged children are out of school due to crisis, poverty, or discrimination (UNESCO).

The Luminos Fund is tackling this challenge head-on, working alongside governments and community-based organizations to run accelerated education programs that help children catch up to grade level, reintegrate into government schools, and prepare for lifelong learning. In just 10 months, Luminos students progress from not recognizing letters of the alphabet to reading short stories. To date, Luminos has helped over 277,787 children in Ethiopia, Ghana, Lebanon, Liberia, and The Gambia secure a second chance to learn.

Organization’s Role & Strength

The Luminos Fund was created to ensure all children have equal access to joyful, foundational learning, especially those shut out of education by crisis, poverty, or discrimination. Our vision is of a world where no child is ever denied the chance to learn. Luminos believes that learning to read is a crucial milestone in every child’s life. At Luminos, we unlock the light in every child through the transformative power of foundational learning. We do this by upholding the following core beliefs and values:

• We believe that every child is capable of learning a remarkable amount in a short period of time, if given the chance.

• We keep children’s joy and well-being at the heart of everything we do.

• We embrace assessment as a key component of effective teaching and learning.

• We celebrate and empower local leadership throughout our work with partner communities and governments.

• We use research, program, data, and skilled classroom observation in the tenacious pursuit of excellence.

• We celebrate the unique contexts and cultures of the communities we serve in our curricula and pedagogy.

• We act with the highest standards of integrity and care, ensuring mutual accountability among colleagues and partners.

• We take the initiative to solve problems where we find them, managing details large and small with urgency.

• We act deliberately to ensure that our organization is inclusive for people of different genders, racial backgrounds, ethnicities, sexual orientations, religious beliefs, abilities, and other sources of diversity.

• We build deep, authentic relationships with our supporters in celebration of the mutually transformative power of giving.

In just one school year, we teach students to read and do math – to learn how to learn – through a joyful, activity-based curriculum. Results of a IDinsight randomized controlled trial (RCT) of the Luminos program in Liberia prove children in Luminos classrooms learn a remarkable amount during the program. In one year, a child in the Luminos Liberia program learns 90% of what the average Liberian will learn in their lifetime.

Furthermore, 90% of Luminos students advance to local government schools to continue their education, and Luminos students are twice as likely to complete primary school than their peers.

Need Summary

Luminos has a robust evidence base and internal and external evaluation system to inform the continuous improvement of our program. Nonetheless, Luminos recognizes the need to strengthen its capacity for data-based decision-making to drive ever-better outcomes for students. For instance, in Luminos programs, average learning gains are significant; however, there is a small but persistent segment of students who seem to attend class regularly but fail to make material progress.

To that end, Luminos is working diligently to strengthen and refine its assessment and data collection capacities to better monitor individual student progress and learning gains in real time. To further improve the learning outcomes and understand the unique learning needs of each child, Luminos must ensure the effective and rigorous collection of high-quality data at the student level, with rapid feedback loops.

The Luminos MEAL system in each country includes weekly teacher-led assessments; thrice-annual curriculum-aligned summative assessments; and weekly field supervisor-administered assessments for a small sample of students in each classroom, followed by classroom coaching to teachers to ensure they are well-equipped to use assessment data to guide learning. This system is supported by Luminos’ network of community-based organization partners. Luminos partners collect individual student assessment data quarterly, which is then processed and analyzed by our country teams to inform programmatic decisions.

There is significant value in streamlining the collection of student-level data, across Luminos program countries, to both increase data quality and further strengthen Luminos capacity to deliver rapid feedback and real time program iteration. Luminos is exploring high-impact, low-cost solutions that will provide further insights into learning variability and inform improvements to teaching and learning strategies through the collection of reliable, localized assessment data.

This low-tech, data-driven solution would allow Luminos to monitor incremental learning gains through weekly student assessments. Luminos works in challenging, low-resource contexts; ensuring the data it collects is relevant, timely, accurate, and complete is thus a key challenge.

Solution summary & next steps

Through the 2023 LEAP Challenge, Luminos aims to pinpoint struggling students earlier in the school year in order to provide targeted support and gain insights into the root causes of these difficulties. This will potentially allow for the creation of risk profiles that predict future needs.

This report serves as a springboard for developing a more robust assessment system that can effectively identify struggling students early on and inform targeted interventions. The recommendations aim to build on existing infrastructure, tools, and processes, rather than develop an entirely new approach to assessment.

The project has three deliverables:

DELIVERABLE 1. The team is proposing a set of guidelines for developing effective assessment strategies that incorporate two complementary approaches.

Using Response to Intervention (RTI), which is a framework that is used in schools to support students who are struggling either academically or behaviorally. The overarching aim is to identify struggling students as early as possible and then to provide them with intervention that is appropriate for their level.

Exploring a Curriculum-Based Measurement (CBM) which refers to a method to track progress toward educational goals, using short indicators of academic performance. CBM involves setting annual goals, frequent assessments, graphing scores, and using data to inform teaching decisions. It can be also used as a component in an RTI approach.

DELIVERABLE 2. The research team created a set of recommendations to identify certain conditions before and during the program that may affect a child's ability to learn at the same pace as other children. These conditions may involve previous schooling or learning opportunities, family literacy and numeracy skills, and age. Timely identification of children who may struggle during the program could allow Luminos to act early, even before the school year, or explore different teaching strategies.

DELIVERABLE 3. At the request of Luminos, the research team suggests options to conduct one or more pilots that will allow them, together with their implementing partners on the ground, to test the effectiveness and efficiency of deliverables 1 and 2.

Introduction

Despite large increases in school enrollment rates around the world, nine out of ten children in sub-Saharan Africa can not read a simple sentence by the end of primary school. Kenya Connect supports children’s literacy development through three primary programs in Kenya, including a school library program, literacy clubs for children, and a livelihood program for mothers. Evidence-based literacy programs are crucial to address this urgent need so that children have foundational literacy skills to succeed at school and in life.

The aim of this LEAP project was to support Kenya Connect with recommendations drawn from best practices in order to enhance their programs with a comprehensive research and funding plan as well as to generate evidence on and support for Kenya Connect’s programs.

Organization’s role & strength

Kenya Connect's mission is to empower students in rural Kenya for success in the 21st Century through breaking down barriers to education and providing educational enrichment and empowerment programs. Improving literacy skills and fostering a love of learning have been at the forefront of their work. They achieve this through three school-based programs - School Libraries, LitClubs, and LitMoms. These programs integrate digital and physical literacy resources, provide training for mothers, and emphasize the joy of reading, making a lasting impact on children's reading habits, engagement in learning, and social and emotional well-being.

The organization’s mission is to “engage and empower students and teachers in rural Kenya to succeed in the 21st Century.”

Need summary

Kenya Connect is committed to evidence-based practices and seeks to collaborate with researchers to enhance the impact of their programs for schools, children and families through its dedicated initiatives. The organization has identified several funding and research priorities. A primary focus of the LEAP project was to highlight Kenya Connect’s core strengths relevant to children's outcomes, determine its unique features in the broader landscape of literacy initiatives in Kenya, and identify the key components of the programs. From this, a plan to document evidence of impact was developed. Considering the high costs associated with research, a significant goal for the organization has been to establish a practical and sustainable funding plan for research and implementation activities. The LEAP project also aimed to assist with the development of the funding plan and align it with the recommended research activities.

Solution summary & next steps

The Fellows extensively mapped the activities in each of the programs that Kenya Connect runs to articulate their unique proposition and supplemented this with direct testimonials from program participants, independently obtained by our Fellows. This provided anecdotal evidence of effective practices and helped to highlight the voices of program beneficiaries (Deliverable 1). Subsequently, we crafted an evaluation plan rooted in a thorough mapping of all Kenya Connect activities and tailored to the organization’s need for evidence-generation for the focal programs (Deliverable 2). This plan includes a proposed research design (both impact and process evaluations), as well as suggested measurement tools aligned with key outcomes targeted by the programs based on Deliverable 1. As mentioned, the focal programs include the School Libraries, LitClubs, as well as the supplementary value of LitMoms, through a quantitatively-focused study. In Deliverable 3, we charted the funding strategy currently pursued by Kenya Connect and aligned it with potential funding avenues pertinent to the activities and unique value of Kenya Connect in the field. This information was integrated with the research plan to ensure research activities align with and inform fundraising priorities, as well as additional evidence-building strategies, into an overarching evidence strategy document for Kenya Connect (Deliverable 4). This final strategic document will guide the organization's forthcoming evidence-building efforts, and its methodology can be tailored by other entities in the sector to aid charities and NGOs in their evidence-building journey.

Introduction

The level of Braille literacy rate around the world is extremely low at less than 10% worldwide, and less than 1% in India. Thinkerbell Labs has a unique product (Annie) that teaches braille literacy - it is the world's first self-learning braille device.

Organization’s role & strength

Founded in 2016, Thinkerbell Labs is a company based in India who specialise in braille literacy. They are a tech-driven organisation building innovative, accessible solutions in efforts to make education inclusive. Their current focus is to make products that help improve learning outcomes for the visually impaired. They have developed Annie - a self-learning Braille literacy device that helps the visually impaired learn to read, write, and type in braille through multimodal content.

Need summary

Thinkerbell Labs is a tech company from India specialising in braille literacy. They seek to understand the metrics that can be tracked via Thinkerbell's analytics platform Helios.

LEAP Fellows created a report where they outlined evidence around braille literacy and provided steps that Thinkerbell Labs can take in order to develop a comprehensive research strategy, especially regarding assessing braille literacy and validating and further developing benchmarks for braille literacy in India. Finally, Fellows led a Theory of Change workshop to help Thinkerbell build a comprehensive TOC.

Solution summary & next steps

The Thinkerbell Labs team is advised to allocate substantial time to read and comprehend the report Fellows provided, the Summary of the Literature Review as well as the detailed research papers cited within. This foundational understanding is essential before progressing to actionized subsequent next steps as outlined in the report.

Introduction

In recent years, the Early Childhood Development (ECD) agenda has garnered significant attention from governments, civil society and business. The recognition comes from a realisation that to thrive later in life, the right foundations are essential and not optional. In South Africa, poverty, unemployment and the rural-urban divide continue to be significant barriers to accessing basic services including ECD. It has become apparent that different modalities are required to respond to the needs of families with different socio-economic circumstances. As such, Seriti’s aRe Bapaleng programme has, for five years, provided basic capacity-building to parents and caregivers of children aged 0-8 years with the intention to empower them to assume the role of the first teacher — providing home-based early learning and stimulation using minimal and inexpensive resources. Through aRe Bapaleng’s Active Learning Workshop methodology, cohorts of parents and caregivers are trained on six sessions which provide relatable information of various aspects of child development including health and nutrition, stages of development, the importance of learning through play and psychosocial support, among other topics. To complement this information, parents and caregivers are also provided with an activity book, a set of Lego Six Bricks, flashcards, and story books to demonstrate the types of learning activities they can implement with their children. Additionally, the programme has implemented other interventions including home-based visits and awareness campaigns to strengthen the support provided to families and communities.

To date, aRe Bapaleng has reached 52,665 parents and caregivers through its interventions and has indirectly benefited 102,534 children.

Organisation’s role & strength

Seriti Institute, established in 2009, is a non-profit and public benefit organisation focused on building resilient communities through innovative and sustainable solutions. Working collaboratively with social partners, Seriti delivers technical support, programme management, and implementation services to help vulnerable people achieve self-sufficiency and create meaningful lives.

Seriti's aRe Bapaleng programme demonstrates a unique value proposition in the South African ECD space through its holistic, community-centred approach, emphasising the importance of home-based nurturing care and household resilience in advancing educational outcomes for children aged 0-8. The programme empowers and equips parents and caregivers with essential skills while building sustainable local ECD ecosystems through active learning workshops, caregiver networks, and strategic partnerships that foster nurturing environments for children's development.

Over the past five years of successful implementation, and three Partner Networking Events aimed at building linkages within the South African early learning and development ecosystem, Standard Bank South Africa remained a steadfast pillar of support. Seriti’s aRe Bapeleng programme has advanced significantly and made it through to the 2024 LEAP Challenge supported by MIT and the Jacobs Foundation. Seriti’s aRe Bapeleng programme demonstrated global relevance, becoming one of ten global finalists to participate in the LEAPathon. Through LEAP, Seriti has been able to strengthen the aRe Bapaleng programme’s Theory of Change, Intervention Design, Data Collection Strategy, and strategic thinking around fundraising in the South African ECD, parental support and household resilience landscape. The LEAP challenge has provided access to research and social enterprise thought leadership at a global level, embodying a strength for continuous learning for future program iterations. As Seriti enters 2025, aRe Bapaleng is well-placed to innovate and attract enduring forms of support for this important intervention in the lives of parents, caregivers and children.

Need summary

The Seriti Institute shared several needs and areas for exploration related to their aRe Bapaleng early childhood development programme:

• A review of their Theory of Change involving key indicators, their multiple interventions, and their interconnections (Deliverable 1)

• A set of recommendations to strengthen their intervention design based on a literature review (Deliverable 2)

• A proposed approach to their data collection strategy aiming to enhance their monitoring, evaluation, and learning processes (Deliverable 3)

• Key recommendations for their fundraising strategy connected to the reflections of the overall project (Deliverable 4) Key recommendations for their fundraising strategy, based on insights from the mapping and landscaping of ECD organizations, emphasize the importance of diversifying funding sources, mitigating donor cannibalization, and positioning aRe Bapaleng’s unique household-focused interventions as a key driver for long-term impact and sustainability.

Solution summary & next steps

The Seriti Institute’s engagement with the LEAP initiative has resulted in actionable insights across four key deliverables, aiming to enhance the aRe Bapaleng programme’s impact and sustainability:

• A refined Theory of Change now sharpens the focus on core interventions and parent and caregiver outcomes while ensuring adaptability and clarity in programmatic objectives.

• Evidence-based recommendations on strengthening the aRe Bapaleng intervention design highlight the potential value of combining group workshops and home visits, optimizing intervention dose, and emphasizing the role of skilled providers and father involvement.

• A proposed data collection strategy outlines insights and considerations for incorporating a Likert-scale approach and emphasizing parental self-efficacy as a primary outcome measure, while also introducing the potential for child outcome tracking using validated tools.

• Lastly, an analysis of Seriti’s fundraising strategy and positioning in relation to other similar interventions based on insights from the mapping and landscaping of ECD organizations, emphasizes the importance of diversifying funding sources, mitigating donor cannibalization, and positioning aRe Bapaleng’s unique household-focused interventions as a key driver for long-term impact and sustainability.

Moving forward, the Seriti Team may consider several next steps, guided by their organizational priorities, funding, capacity, and strategic goals. These include:

• Piloting and institutionalising the refined Theory of Change and intervention designs and developing a set of accompanying key indicators

• Incorporating revised data collection tools to measure parental self-efficacy and child outcomes

• Exploring opportunities to scale program components through strategic partnerships, aligning their narrative to funder interests, and showcasing the long-term societal benefits of their work to ensure sustainability and resilience in their impactful ECD interventions.

Introduction

Education for Sharing (E4S) is an international organization that forms better global citizens through innovative education. E4S is committed to providing meaningful experiences for all, using a unique methodology that harnesses the power of play, reflection and people's inherent ability to shape their own reality. Established 17 years ago in Mexico, E4S has worked with over 1.8 million beneficiaries from the educational community across thirteen countries.

Since its foundation, E4S has developed ten educational programs directed to a wide array of ages with different objectives aligned to its mission. One of the newest is called Grow for Sharing (G4S) and is focused on early childhood.

G4S is an educational program that builds global citizenship from childhood through play, teaching values like fair play, empathy, and teamwork. Focused on children ages 3 to 6, it uses hands-on activities to develop lifelong skills such as curiosity, collaboration, and critical thinking, fostering confident, socially responsible individuals.

Organization’s role & strength

Mission

E4S’s mission is to form better global citizenship from childhood through the power of play. To achieve this, they incorporate core values of fair play, gender equality, tolerance, respect, empathy, responsibility, and teamwork into all aspects of their work. The objective is to change the way we look at education and youth development and, through local community efforts, drive systemic change.

Vision

To serve as a world reference in shaping global citizenship and cultivating a sense of agency starting from early childhood.

Need summary

The G4S team needed to understand which methods best instill positive behaviors and knowledge in preschoolers and how to measure these impacts effectively from the child’s perspective. They aimed to identify the most reliable assessments, like caregiver interviews or teacher evaluations, for comparable results.

Solution summary & next steps

Following a few discovery sessions with G4S and reviewing their organizational objectives, we suggested the following:

• Pre-deliverable: Visual representation of the G4S program

• Deliverable 1: Literature Guide

• Deliverable 2: Assessment for children

• Deliverable 3: Recommendations on MEAL Plan components

• Deliverable 4: Future-focused Roadmap

Introduction

STS International’s Whole Child Model program in Tanzania includes the implementation of flexible ability grouping in Tanzanian schools.

The flexible ability grouping intervention aims to enable teachers to identify and support struggling learners in classrooms with a high number of students, reduce assessment time and inform systematic next steps in guiding the student learning journey.

While STS has gathered evidence of positive feedback from teachers for the ability grouping approach in schools and created an initial toolkit for teachers to use, before scaling the program up to district levels or beyond, there remain challenges to the successful implementation and completion of ability grouping in its current form that STS seeks to understand and improve.

These challenges include but are not limited to effective grouping and decision making based on teachers’ analysis of data in the ability grouping process, as well as the consistent implementation of ability grouping using the recommended tools during lesson time.

Organisation’s role & strength

STS has two roles. First, STS/International (STS/I) assists governments in low- and middle-income countries to assess student learning, train teachers, and develop educational policies that support improved learning. Second, STS/Tanzania is a local initiative of STS/I, where STS is piloting the Whole Child Model (WCM), combining support for education, health, and parent and community engagement so that all children can succeed in primary school. One component of the WCM is flexible ability grouping (AG), in which teachers in STS-supported schools are trained to organize their students into learning groups, assess them while in their groups, then tally the results to determine which students are still struggling. STS is the originating designer of this form of AG and provides basic implementation protocols and tools to teachers to use in the practice.

While uptake of AG has been high, even in class sizes of 100 students or more, many teachers have struggled to understand the needs of individual students and provide appropriate remediation. STS thus sought the assistance of LEAP for two reasons: (1) to identify areas where the existing AG process can be streamlined to make it more feasible to execute faithfully for all teachers, and (2) to identify strategies that could help teachers assess the needs of students who are struggling most, even in large classes, so those teachers can provide appropriate support to those struggling students. The thought is that the AG approach already instituted by STS can be used as a “first pass” to assess all students, even in large classes, and that a second, more focused strategy can be found to address the needs of students found to be struggling most.

Need summary

In order to assist in improving the fidelity of implementation of flexible ability grouping, the LEAP fellows have taken the approach of identifying a relevant base of research through a literature review, identified the need to clarify the implementation steps within the ability grouping protocol, as well as understand from teachers applying their flexible ability grouping training, what the key resources that they require in their classrooms are to narrow the scope of execution for the most valuable tool or tool framework to be created.

Solution summary & next steps

Literature Review

The literature review highlights the potential and challenges of ability grouping to address diverse learning needs in large classrooms, such as those in Tanzania. This approach to differentiated instruction includes homogeneous ability grouping, mixed-ability grouping and flexible ability grouping.

The flexible ability grouping approach organizes students into fluid, temporary groups based on specific learning objectives, allowing teachers to provide targeted support. Unlike the more “permanent” ability groups, flexible grouping mitigates issues of social comparison and fixed expectations by enabling students to move between groups as their understanding progresses.

Researchers and practitioners note that the effective implementation of flexible ability grouping requires frequent formative assessments to monitor student progress, enabling teachers to adjust instruction responsively. However, teachers face challenges in managing these groups, especially in large classes, without adequate training, planning time, and resources.

The literature review concludes that to maximize the benefits of flexible ability grouping, Tanzanian classrooms will need to adapt or introduce support structures for ongoing assessment, professional development to manage group dynamics, and resource allocation for planning and differentiation.

The insights of our literature review, with input and knowledge of the STS team formed the basis for our subsequent teacher discovery questionnaire to identify tool or framework creation priorities.

Protocols and Tools

• Our initial discovery work into the steps and stages of the ability grouping process have identified a multi-stage process where particular stages can be a road-block or potentially confusing to an implementing teacher.

• We have created detailed visual aids to assist in making next step decisions more intuitive and take less time for teachers, accompanied by a teacher guidebook to detail the process in a referenceable format during lessons.

• We have identified that teachers struggle to have sufficient planning time and in particular to access content and differentiated activity and assessment resources. We have created an open content channel with free resources as well as constructed an installable apk suite from a recognised content package that has been tested with Tanzania students in Swahili.

Next Steps and Conclusions

STS’s model of ability grouping serves as a means to identify and provide an equitable opportunity to advance learning progression across the spectrum of students in the extraordinarily high student teacher ratio operating environment of a Tanzanian classroom. However, ultimately the model seeks not only to identify these students, but to take further steps for the most at-risk students to receive individualised support from empowered and enabled teachers.

We have identified and overviewed in our report that there are several initiatives successfully furthering progress in both supporting the ongoing learning, as well as the individual assessment of primary school students in Tanzania, both in-school and in an out of school time setting. These initiatives appear complementary to the work that STS is doing, however rely on student facing technology devices within classrooms. Until this can be made possible, access to these opportunities will continue to be limited.

We have concluded that the most important focus for the STS Ability Grouping program is to continue its focus on developing its fidelity of implementation, whereupon a greater consequent scale in the program’s execution will lead to the fundraising opportunities to catalyse technology access.

About Street Child UK (SCUK)

Street Child works with local organisations to ensure every child has access to an education. Their purpose is to see all children safe, in school and learning – especially in low-resource environments and emergencies.

Street Child in Somalia

African Education Trust (AET), part of Street Child, started working in Somalia in 1996, at a time when many organizations were leaving the country after the aborted UN Peacekeeping mission. By working through local staff, organizations, and institutions they have been able to continue to operate throughout the country despite successive cycles of drought, famine, and conflict. They provide essential quality education through school rehabilitation, psycho-social support for children experiencing long-term insecurity, teacher training, supporting parental involvement in school management and provision of hygiene facilities especially for girls and children with disabilities.

Context

Somalia and Somaliland face significant educational challenges, including low literacy rates and 4.8 million children out of school with 1.4 million children living in IDP (internally displaced) settlements. Most internally displaced children have little to no foundational education.

Street Child's initiative to improve foundational learning outcomes through the Teaching at The Right Level (TaRL) approach aims to address these issues in both regions. The TaRL focuses on assessing children’s foundational literacy and numeracy skills and grouping them by ability rather than age and teaching is targeted at the child’s learning level. This method is particularly useful in low-resource settings with diverse learning levels.

Project Host Need and Deliverables

LEAP Challenge Fellows were pleased to partner with SCUK to address challenges related to scaling their TaRL program.

Having recently concluded an initial pilot in Somalia and Somaliland, SCUK sought to determine how to adapt the programme for different settings, identify which programme components were scalable, and manage costs while maintaining programme quality. SCUK approached the LEAP Challenge Fellows with a number of broad questions:

• What works?

• What can be scaled/replicated?

• Utilisation of NGOs and/or government for sustainability?

• How effective is TaRL in a highly conflicted and fragile environment?

• Can we use a stepped scale model in Somalia and a mixture of bridging and remedial approaches?

Considering these questions, as well as opportunities recently presented to SCUK, we identified three core deliverables to address their needs:

1. A Flexible Implementation Model to adapt the programme across contexts.

2. Education Scalability Checklist to consider the feasibility of scaling the programme from the perspective of key stakeholders.

3. A Budget Analysis Tool to simulate various budget scenarios and compare cost effectiveness across different implementation approaches.

These deliverables are designed to help SCUK make informed decisions about scaling the TaRL programme sustainably while also serving as resources that other organizations can adapt for similar challenges.

How We Worked Together

LEAP Fellows, Annelise Buzaid, Drew Edwards, Melissa Hogenboom, and Simi Lawoyin, partnered with the Project Host, Street Child UK, over a 12-week period to learn about their programme and co-create solutions to support their scaling endeavours.

• Collaboratively

  We worked closely with the Project Host to understand their goals, challenges, and priorities. Through a series of working sessions, we ideated, built out initial drafts and refined them together to align with SCUK’s vision for the project.

• Iteratively

  At every stage, we gathered feedback from the SCUK team to ensure our deliverables were relevant and feasible to implement. This iterative process allowed us to make real-time adjustments and ensure alignment with SCUK’s operational realities.

• Leveraging Existing Frameworks and Tools

  To benefit from proven methodologies and maximize efficiency, we used existing frameworks like the Brookings Education Scalability Checklist and a proprietary stakeholder engagement matrix (developed by one of the Fellows). These tools provided structure for our deliverables and ensured that the solutions were rooted in established best practices while tailored to SCUK’s specific needs.

In approaching this project, we wanted to ensure the practicality of the tools we developed, making them user-friendly and actionable for SCUK's needs. We also had in mind the need for adaptable solutions that other organizations in the education sector could use to address similar challenges.

Introduction

Arukay is an EdTech company dedicated to equipping students in Latin America with essential digital skills, computational thinking, and coding expertise to prepare them for the demands of a technology-driven world. By integrating digital literacy into K-12 education, Arukay aims to break cycles of poverty and create opportunities for future generations. For over a decade, Arukay, as a growing organization has impacted over 100,000 students across multiple Latin American countries, Arukay seeks to strengthen the evidence base of its educational model through a LEAP Project with MIT Solve.

Organisation’s role & strength

Arukay operates as a for-profit EdTech company with a mission to transform education by making digital literacy accessible to students, regardless of socioeconomic background. The organization is led by CEO and co-founder Vicky Ricaurte, whose expertise in management and educational technology positions the company as a leader in the field. The team includes specialists in curriculum development, platform engineering, finance, and commercial outreach, ensuring cross-functional collaboration and strong implementation capabilities. Arukay’s success is driven by its structured and dynamic instructional design, robust teacher training programs, and advanced data analytics for learning assessment.

Need summary

Latin America faces a critical digital skills gap, with only 17% of students advancing to higher education and a slow intergenerational economic progression. The lack of systematic, early digital education exacerbates social inequalities and limits economic mobility. Arukay seeks to address this challenge by integrating coding and computational thinking into primary and secondary education. However, to maximize impact, the organization requires a stronger evidence base to validate its effectiveness, refine assessment methodologies, and develop scalable measurement frameworks.

Solution summary & next steps

• High-Quality Curriculum: Age-appropriate, multilingual digital literacy courses.

• Teacher Training & Support: Equipping educators with the necessary tools to integrate computational thinking into their classrooms.

• Advanced Reporting & Analytics: Providing real-time insights into student progress.

The next steps include leveraging the LEAP Project to:

1. Improve the design of formative and summative assessments.

2. Enhance data collection methodologies for measuring soft and hard skill acquisition.

3. Develop a long-term framework to track computational thinking skills across primary and elementary education.

Introduction

In Ghana, students often describe their education as "Chew and pour, pass and forget," reflecting a system heavily reliant on rote memorization, particularly in STEM subjects. To address this, the Government of Ghana has set an ambitious goal of having 60% of students pursue STEM fields at the university level. However, this shift requires a transformation in how STEM education is delivered.

The Practical Education Network (PEN) is at the forefront of this change, empowering educators to create hands-on, inquiry-based learning environments using locally available, low-cost materials. PEN’s innovative approach bridges the gap between theory and practice, enabling students to engage with STEM subjects in a tangible, interactive way. To date, PEN has trained around 150 trainers and 9,000 teachers, reaching 2 million learners who have benefited from meaningful, hands-on STEM education for the first time.

The LEAP project aimed to support PEN in three critical areas: designing a tracer study to evaluate long-term program impact, providing recommendations for organizational growth, and developing strategies to contextualize and communicate program effects effectively.

Organisation’s Role & Strength

PEN’s mission is to enable every African child to learn by doing. They are transforming STEM education in Ghana and beyond by equipping educators with the tools and training they need to implement practical, experiential learning. The organization’s innovative teacher training program reduces barriers to hands-on learning in material resource-constrained settings, fostering critical thinking, creativity, and problem-solving skills among students.

PEN’s impact extends beyond classrooms: its methods have been integrated into Ghana’s national science curriculum and it has helped bring “hands-on” and “STEM” into everyday parlance amongst education stakeholders in Ghana. While the organization extends its reach geographically, it is also interested in understanding the depth of its impact on students. Specifically it seeks to uncover the extent to which students taught by teachers trained in hands-on approaches may be more likely to pursue STEM majors, which can directly inform policies to achieve the government’s goals. PEN’s emphasis on sustainability through the reliance on locally-available materials and on scalability through the training-of-trainers model positions its approach to be replicated across diverse educational contexts.

Need Summary

To achieve its mission, PEN seeks to deepen its impact through robust research, improved program fidelity, and effective communication of results. Key priorities include:

Evidence Generation: Developing a tracer study to track the long-term impact of its teacher training programs on student outcomes and career choices.

Program Fidelity: Strengthening implementation practices to ensure consistent quality across training sessions and classrooms.

Growth & Communication: Expanding digital resource libraries, fostering community forums for teachers, and refining strategies to communicate program outcomes to stakeholders, including government officials and funders.

The LEAP project focused on addressing these priorities through tailored recommendations and actionable plans.

Solution Summary & Next Steps

The LEAP Fellows provided PEN with four key deliverables to support its goals:

1. Tracer Study Plan: Fellows designed a comprehensive tracer study to evaluate the long-term outcomes of PEN’s programs on teachers and students. This included developing sampling procedures, measurement tools, and analysis strategies to assess teacher adoption of hands-on methods and student progression into STEM fields (Deliverable 1).

2. Organizational Growth Recommendations: Fellows proposed strategies to improve program fidelity, such as incorporating a conceptual framework for measuring implementation consistency. Additional recommendations included expanding PEN’s digital resource library and fostering online community forums to facilitate peer learning among educators (Deliverable 2).

3. Future Research Options: Fellows outlined research opportunities to further PEN’s understanding of the impact of their program. This included a detailed typology of studies to assess the impact of PEN’s programs and strategies for aligning these studies with organizational priorities (Deliverable 3).

4. Contextualizing & Communicating Effects: Fellows provided a framework for effectively communicating PEN’s impact to diverse audiences, including government stakeholders. This included recommendations on using standard deviations to present effect sizes and models for clear, impactful storytelling (Deliverable 4).

These deliverables collectively will equip PEN to enhance its program delivery, generate evidence of its impact, and scale its approach to transform STEM education across Ghana and beyond. The methodology developed through this project can also serve as a model for other organizations seeking to implement and evaluate hands-on STEM education programs in settings with minimal material resources.

Introduction

DECILE is a language evaluation tool, created by CEPE at the Di Tella university in Buenos Aires. It assesses the language skills of Spanish-speaking children with a hearing impairment aged 3 to 12 and is also suitable for neurodiverse populations.

By evaluating children's language attainment through simple tasks, it aims to provide educators with individual student metrics so they can make informed decisions about students’ pedagogical paths. Still in the pilot phase, DECILE has been evaluated with over 200 children with and without hearing impairment in Buenos Aires.

Although English language assessment tools exist, there is currently no tool designed to test Spanish linguistic structures and dialects. DECILE aims to become the gold standard tool for assessing the language skills of Spanish-speaking children.

The DECILE team at CEPE Di Tella are engaging in the LEAP project to strengthen their solution and prepare to scale it.

Organisation’s role & strength

Based at the School of Government at Universidad Torcuato Di Tella, the Center for Evaluation of Policies and Evidence-Based Policy-Making (CEPE Di Tella) specializes in conducting applied research to improve the quality of public policies through the generation and use of rigorous evidence.

CEPE Di Tella’s Education and Behavioral Sciences Program focuses on the analysis of educational policies using social, pedagogical, economic, behavioral, and neuroscientific determinants of learning, with a particular emphasis on promoting equity and expanding opportunities for children from underserved populations.

Since 2017, CEPE Di Tella has collaborated with schools in Argentina to support the education of Deaf and Hard of Hearing (DHoH) children. Within this line of work, DECILE plays a central role as an innovative tool for assessing language development in Spanish-speaking children, addressing a critical gap in available assessment instruments for this population.

DECILE enables large-scale, evidence-based monitoring of linguistic abilities while collecting key demographic and environmental variables that influence language development. The tool is designed with scalability in mind, aiming to inform educational practices and policy at a national and regional level, while providing personalized recommendations for educators, therapists, and families.

The development of DECILE reflects CEPE Di Tella’s interdisciplinary approach, combining expertise from education, behavioral sciences, linguistics, cognitive neuroscience, and public policy to advance both scientific understanding and practical interventions that foster inclusive, high-quality education.

Need summary

The DECILE team identified four priority needs for the LEAP project:

1. Limited Validation & Benchmarks: DECILE has been tested with a small user base, making its reliability and validity uncertain, especially in the absence of benchmarks for Spanish-speaking children.

2. Scalability & Adoption Challenges: While the team aspires to scale DECILE, there is no clear strategy for broader adoption, and its current design does not support independent use in classrooms.

3. Low Practical Value for Teachers/ Students: DECILE lacks a clear in-class usage protocol and does not yet provide meaningful insights or impact for teachers and students.

4. Technical & Implementation Barriers: The platform’s current capabilities do not support large-scale, independent use by educators, limiting its effectiveness beyond a research setting.

Solution summary & next steps

To address these needs, the LEAP project has created four corresponding deliverables.

1. A literature review of similar language assessments, and guide for psychometric analyses, created through reviewing secondary research and mapping the landscape of Spanish language assessments.

2. A Growth Toolkit, containing a growth roadmap for the next five years, a landscape analysis summarising lessons from similar products, an overview of key growth levers, a go-to-market strategy and tools to support effective scaling.

3. A proposal for a teacher-facing insights dashboard, co-designed with teachers, developed following a review of best practices for dashboard development.

4. Proposed UX flows for different stakeholders to ensure that teachers, students, and other key stakeholders can use the app in a self-directed way

As a result of these LEAP project deliverables, DECILE will be able to:

1. Expand Validation & Benchmarking of DECILE with a larger, more diverse student sample and align it with best practices to establish reliability, validity, and effectiveness.

2. Strategically prepare for growth by refining the tool for wider implementation across Latin American schools, addressing diverse needs.

3. Plan technological improvements to support independent, large-scale use by teachers (incl. feedback dashboard, usage protocols).

These deliverables can also serve as a public good, with its tools and insights relevant both for others developing linguistic assessments or looking to scale a research-based ed-tech product.

Introduction

Fundación Aprender a Quererte (FAAQ) is a new philanthropic platform dedicated to enhancing educational opportunities and fostering economic mobility in Colombia. Over the next few years, FAAQ aims to become an incubator for Colombia’s public education sector—identifying, adapting, and rigorously testing educational solutions before partnering with the government to achieve scale.

Organisation’s role & strength

FAAQ’s unique value proposition lies in its ability to rapidly test and refine education solutions, ensuring they are relevant to local needs, feasible to implement and scale, and effective in helping kids learn.

FAAQ's flagship program, Enseñar al Nivel Adecuado (ENAd), is an accelerated learning initiative aimed at ensuring that all students master foundational literacy and numeracy skills before they transition out of primary school. This program brings Pratham's Teaching at the Right Level (TaRL) approach—successfully implemented across Africa and Asia—to Colombia and Latin America for the first time.

Need summary

Through the ENAd program, FAAQ has reached 3,500 children in Bogotá and 4,800 in Risaralda. To monitor, evaluate, and learn from these pilots, FAAQ has adapted several of Pratham's research instruments and developed additional tools to provide a more holistic view. This data has allowed FAAQ to track student and teacher progress and respond quickly when needed. At the same time, FAAQ’s current MEL system revealed several gaps, particularly in data generation and use, which were validated by the Jacobs Foundation's Evidence Navigation Journey (ENJOY) framework. To address these gaps, the Fellows prioritized the following questions to explore during the LEAP Sprint.

1. Based on the available ASER data, what insights can be gained regarding the effectiveness of the ENAd program? How can these insights inform the content and implementation of the program?

2. What are the limitations of the current data and the open questions that remain? How can data collection be refined to address these gaps?

3. How can data collection and utilization processes be optimized and systematized to enhance the efficiency and effectiveness of data-driven decision making?

4. What specific guidelines and templates can be developed to enhance the consistency and usability of data collection, analysis, and reporting?

Solution summary & next steps

Deliverable 1:

This deliverable provides FAAQ with concrete advice on how to grow their ambition to become the leading Educational Solutions Lab’ In Colombia. To be an effective education solutions and evidence lab, it is essential to use data and insights for continuous improvement and informed decision-making. This involves actively generating evidence that helps FAAQ and others adapt programs to the unique needs of Colombia. Deliverable 1 provides guidance on analyzing and visualizing current ASER data, as well as on enhancing future data acquisition to generate actionable evidence. Specifically, Deliverable 1 provides detailed guidance on different ways to analyze and visualize data that FAAQ has previously collected during their implementation of their ENAd program. The analyses, findings, and recommendations that are presented are meant as an example that can be transferred to other data sets that FAAQ may be collecting in the future and they grow as an Educational Solutions Lab’.

Deliverable 2:

Deliverable 2 provides FAAQ with a practical toolkit to strengthen data use across the program cycle. It consists of two integrated tools: the Evidence-to-Action (E2A) Framework for planning MEL strategies, and the E2A Tracker for documenting data collection, analysis, learning, and follow-through on reporting and taking action. Together, the tools give life to the recommendations in Deliverable 1 by making it easier for staff to collect the right data, ask the right questions, and follow through on what the evidence reveals.The toolkit reflects FAAQ’s emerging identity as an Evidence Lab by supporting systematic, intentional, and adaptive learning at the program level.

Introduction

In Zambia, a staggering 98% of 15-year-olds fail to achieve the minimum PISA-D proficiency level in mathematics (OECD, 2017). Across sub-Saharan Africa, learning poverty rates exceed 90% (World Bank, 2024).

Behind these statistics are tens of millions of young lives with limited opportunities and unfulfilled potential - and with rapid population growth placing increasing pressure on already fragile systems, the region is facing a “growing educational crisis” (World Bank, 2025). In response, UNESCO, African Union, and United Nations Children’s Fund (2025) have called for better educational data that is relevant, accessible, and understandable to stakeholders.

Koloso, a Zambian-owned and operated enterprise, helps to address this need by generating, analysing, and distributing reliable learning data in real-time to teachers, parents, school leaders, and policymakers. As a result, Koloso enables teaching, managing, and directing for improved learning outcomes and catalyzes sustainable, systemic change that delivers positive impact at scale.

The Koloso approach leverages digital innovation through high-frequency, low-impact gamified assessments that take just two minutes to complete but yield powerful insights. These insights flow in real-time to every stakeholder in the educational ecosystem: teachers can target instruction with precision, students can focus their efforts where most needed, parents can provide informed support at home, and policymakers can allocate resources based on evidence rather than intuition.

Currently focusing on primary and secondary mathematics - where the learning crisis is most acute - Koloso aims to become sub-Saharan Africa's most affordable and contextually appropriate formative assessment platform across the entire school curriculum, delivering the data that can transform how education systems respond to the region's most pressing challenge.

Organisation’s role & strength

Koloso has the potential to drive improved quality of education in resource-constrained environments through a simple but powerful approach: converting brief student interactions into actionable insights that can drive change at all levels of the educational system.

At the heart of the system is the Koloso Challenge - a daily quiz game accessible via mobile and web applications. Students have two minutes to answer ten curriculum-aligned questions that are selected based on their grade level, curriculum requirements, and their teacher's instructional plan. This approach generates robust learning data with minimal disruption to classroom time - and can be used as a homework assignment.

What really sets Koloso apart is how this data is transformed into customised insights for every stakeholder in the educational ecosystem:

• Teachers receive real-time, detailed analytics on individual and class-wide learning, allowing them to identify learning gaps immediately and deliver targeted instruction where it's most needed

• Parents gain real-time visibility into their child's topic-by-topic performance, fostering increased engagement in learning, and meaningful home-school partnerships

• School leaders access comprehensive dashboards showing performance trends across classrooms and subjects, facilitating resource allocation and instructional leadership

• Policymakers obtain aggregated, anonymized data that reveals systemic patterns, supporting evidence-based decision making at regional and national levels

This multi-level approach has generated remarkable market validation: within a few months of its commercial launch, Koloso has already secured almost 10,000 users from 25 subscribing schools and NGOs in Zambia and South Africa - indicating substantial demand for its solution.

As the Koloso community expands, the organization’s intention is to evolve from being solely a data consumer (learning from evidence) to becoming a data provider - creating a substantial repository of learning data in Sub-Saharan African contexts that can inform research, resource allocation, and policy development throughout the region.

Need summary

Having gone to market in October 2024, Koloso is in a pivotal phase of development as it learns from its first customers. Koloso needed support interpreting customer feedback to:

• Refine its sales strategy, so that it can continue to grow and attract investment; and

• Prioritise new features and product developments that are most likely to increase student learning: the primary goal of Koloso’s customers.

In particular, Koloso sought to draw on the literature of low-tech digital assessment interventions in the Global South to create a research-based product development roadmap.

Furthermore, Koloso needed a research framework to define the claims it can make with its existing data and to plan for further evidence-generation and evaluation efforts, ultimately equipping Koloso to ensure it’s achieving its vision of eradicating learning poverty in Sub-Saharan Africa.

Solution summary & next steps

Koloso’s engagement with LEAP has resulted in actionable insights across three deliverables: a research-based product development roadmap, a research framework, and a refined sales strategy. Drawing from these deliverables, the LEAP team has agreed the following recommended actions with Koloso:

Research-Based Product Development Roadmap

This deliverable details relevant existing research, highlighting trust, heterogeneity of outcomes, how best to involve parents, and the potential role of artificial intelligence. Key recommendations are:

1. Conduct qualitative interviews with teachers, parents, and/or students to help determine which factors influence participation rates (e.g., trust) among some schools and/or subgroups, which features are the most popular or least popular, and future changes that would help improve the app.

2. Collect data on parent/family/student demographics to determine if there are different effects of Koloso on students with different characteristics (e.g., family income level, student gender, etc.). This can inform changes to the app so that it may best serve diverse groups.

3. Help parents to accurately interpret student performance data to enable an understanding of student learning.

4. Consider developing an AI-guided experience of feedback and quiz problem selection, facilitated with teacher input, to increase student learning.

Research Framework

This deliverable reviews prior research relevant to app-based assessment and intervention. It then provides recommendations to better display existing data, collect new data, and conduct a rigorous assessment of app effectiveness. Key recommendations are:

1. Collect/calculate/present additional data, such as student reaction time, to highlight a different aspect of student performance.

2. Identify individual and group performance on specific questions to highlight potential areas of strength or weakness.

3. Conduct a randomized controlled trial (RCT), quasi-experimental design, or pre-post assessment to make conclusions about the effectiveness of the app on desired outcomes (e.g., teacher preparation time).

4. Incorporate demographic data (from research-based product development roadmap recommendation #2 above) into these analyses to identify and address discrepancies by gender, income, etc.

Sales Strategy

This deliverable consists of stakeholder persona outlines for teachers and parents at local and international private schools in Zambia, as well as a market database with potential customer schools across Sub-Saharan Africa. Key recommendations are:

1. Develop flexible value propositions that address core stakeholder needs across segments:

   1. Teachers: Emphasize how Koloso reduces assessment workload for teachers while providing actionable insights to improve student outcomes.

   2. Parents: Focus on how real-time data enables timely, personalized support for their children.

2. Expand and leverage the customer database as a strategic asset for growth. Expand the database with knowledge of schools in Zambia in the private school market segment and eventually the government school segment. Use this data to identify high-potential prospects, optimize outreach timing, and tailor communications.

3. Utilize the value propositions and database list to refine sales strategies for each respective segment to reach decision-makers effectively. Combine direct engagement with school leadership, teacher workshops, and digital communication channels to build relationships and provide ongoing support to all stakeholders.

4. Prioritize product development based on core user needs while maintaining flexibility. Focus on developing features according to the product roadmap. Then establish a roadmap for expansion to additional subjects beyond mathematics.

Inspiring Teachers is a globally recognised nonprofit organisation committed to empowering educators, especially within low-resource contexts. Its core program Tools for Foundational Learning Improvement, combines integrated structured pedagogy with technology to to enable data-driven teacher support and adaptive program management. This program has two components, an instructional leadership program based on peer coaching and a foundational literacy program.

The Inspiring Reading programme provides structured pedagogy and assessments for foundational literacy (Basic 1-3) to teachers in Ghana. The programme consists of user-friendly lesson guides, integrated student workbooks, and the SmartCoach app, which supports School Leaders and Teachers in running in-school coaching programmes and getting real-time insights on teaching via dashboards, enabling prioritisation and progress tracking, making targeted support and adaptive management possible.

Mission

Their mission is to equip and empowering educators with tools to guarantee world-class learning in African Schools.

Vision

Inspiring Teachers envisions a world in which Governments across Africa equip teachers to guarantee foundational learning and fulfill the promise of education.

Need Summary

The Inspiring Teachers team aimed to ensure their teacher-led 1-1 reading assessment data collection methods generated information that was both valid and reliable in preparation for an upcoming Randomised Control Trial (RCT) study. They also wanted to explore ways to make their work in Ghana more sustainable and deeply aligned with Ghana’s Ministry of Education long-term priorities.

Solution Summary & Next Steps

Following a few discovery sessions with the Inspiring Teachers team and reviewing their organisational objectives, we suggested the following:

• Deliverable 1: Research and Measurement Guide

• Deliverable 2: Assessment Strategy

• Deliverable 3: Analysis of User Feedback & Recommendations

• Deliverable 4: Competitive Analysis

• Deliverable 5: Future-Focused Roadmap