VVOB education for development

Describe the product or program that is the focus of your proposed LEAP project.

In Rwanda, VVOB’s “Learning through Assessment and Data (LEAD)” programme aims to support SDG4 by ensuring that all Rwandan children can actively participate in quality education and successfully progress through primary education. For this, we focus on strengthening the competencies of newly assigned school leaders to effectively use data to improve teaching and adequately address repetition, drop-out and equity gaps in learning outcomes, while being supported by sector and district officials. VVOB does this in partnership with the following government partners in Rwanda: Ministry of Education, Rwanda Basic Education board, University of Rwanda College of Education and National Examination and School Inspection Authority. Together with our partners, the LEAD programme aims to provide targeted professional development to about 1500 (newly assigned) school leaders.

Evidence shows that school leaders play a vital role in creating the conditions for effective teaching and learning. LEAD builds strongly on the foundations laid in school leadership in previous VVOB programmes in Rwanda, whereby more than 1570 school leaders benefited from continuous professional development. It further strengthens the distributed leadership model in which (deputy) headteachers work in close collaboration with others, in particular, the School-Based Mentor, responsible for improving teaching and learning and teacher professional development. By equipping school leaders with the competencies to effectively use data for reflection and plan for improvement at the level of the school, these actors develop the habit to rely on and use data. For example, data are available for use on the status of learning disaggregated by different types of students. Data can also be used to monitor the quality of systems and equity. Regular monitoring, in turn, can reveal changes over time in response to interventions to improve student outcomes, providing additional data for decision-making. For teachers this includes using student’s assessment, to understand the causes of the gaps between learners and for self-assessment. For school leaders this involves the use of data to inform the overall school performance supporting planning and monitoring activities.

The programme consists of the following continuous professional development (CPD) modalities:

• Two certified CPD programmes:
  • the Diploma Effective School Leadership. Target group: head teachers and deputy head teachers.
  • the Certificate Educational Mentorship and Coaching. Target group: School Based Mentors, district and sector education officials.
• CPD support through mentoring and coaching to head teaches and deputy head teachers by sector and district education officials, including through support to peer learning in at two levels:
  • Professional Learning Communities (PLCs) for head teachers and deputy head teachers, facilitated by sector education officials. This learning network supports head teachers and deputy head teachers to implement what was learnt in the Diploma programme and continuously reflect and improve.
  • Communities of Practice (CoPs) for teachers, facilitated by School Based Mentors.

To ensure that newly assigned school leaders can apply effective distributed leadership, they need to be supported by an enabling environment, surrounded by superiors that value and understand their work. For this reason, the sector and district officials are targeted too. Sector officials play a central role in ensuring school leaders can collaborate and learn from one another. Through coaching and mentoring, and facilitation of Professional Learning Communities with school leaders, they can jointly obtain better insights into the challenges that schools face and support one another in taking data-driven decisions. Similarly, district officials, support sector officials in monitoring challenges and opportunities faced by schools and ensuring that these are reported back to the central level.

Select the key characteristics of your target population. Select all that apply.

• Women & Girls
• Pre-primary age children (ages 1-5)
• Primary school children (ages 5-12)
• Youth and adolescents (ages 12-24)
• Rural
• Peri-Urban
• Urban
• Poor
• Low-Income
• Middle-Income
• Persons with Disabilities

In which countries do you currently operate?

• Belgium
• Cambodia
• Ecuador
• Kenya
• Rwanda
• South Africa
• Uganda
• Vietnam
• Zambia

In which countries do you plan to be operating within the next year?

• Belgium
• Cambodia
• Ecuador
• Kenya
• Rwanda
• South Africa
• Uganda
• Vietnam
• Zambia

How have you worked with affected communities to design your solution?

VVOB works together with departments in the Ministries of education and institutions that provide professional development opportunities to teachers and school leaders. Together with these partners, we strengthen the provision of professional development of teachers and school leaders, in turn improving the quality of education for learners from early childhood education up to TVET. VVOB puts the plans of the Ministry partners at the centre of its partnership, providing innovative solutions that support existing structures and systems of the ministries to ensure education systems are equitable and inclusive, and thus provide quality education. Close consultation with our government partners is a precondition for VVOB’s programming activities.

At the level of the beneficiaries, input of school leaders, teachers, government officials and students is sought in several ways. As a standard practice in VVOB’s way of working, a needs analysis with beneficiaries is conducted at the start of an intervention (example of a needs assessment). Also, for the LEAD programme in Rwanda, we are in the process of setting up a needs assessment at the level of school leaders, government officials, teachers, and students. Next to this, we also consult our beneficiaries when evaluating our interventions. The objective of the needs-assessment study is to ensure that the program is in line with the context of primary schools in Rwanda, to inspect the programme theory of change and to identify the key priorities and strategies that guide the implementation of the LEAD programme.  

On a continuous basis, graduates from the CPD programmes in Rwanda also provide their feedback, which in turn is taken up when redeveloping CPD programmes with our partners. Barriers for participation, like limited skills to participate in online learning of school leaders and teachers, are identified and solutions are introduced to overcome these barriers (e.g., offering a preparatory online learning training).

At the level of impact and behavior change of school leaders, a school leadership assessment is conducted among school actors including parents, learners, teachers and school leaders. At the end of the assessment, school leaders are provided with feedback on how they can implement effective school leadership.    

What is your theory of change?

VVOB’s work starts from the premise that if the right to education is to be realised and the promise of Sustainable Development Goal 4 (SDG 4) to be held, then it is necessary to reinforce the capacity of governments to implement key laws and education policies that they have in place. Between the existing legal guarantees and policies and the effective realization of rights and SDG 4 targets lies a major implementation gap, however. Governments are not meeting their obligations as duty bearers and, at least in part, this is due to missing capacity to deliver effective solutions at scale.

As such VVOB focuses on:  

• Reinforcing the capacities of duty bearers to enable them to guarantee the right to education of its citizens (right holders) in a more qualitative and equitable way; 
• Promoting equity (right to non-discrimination) within governments’ education policies and actions.  

If governments and their education development partners want to achieve the right to quality education and reach the SDG 4 targets, then they must invest in quality teachers and school leaders. A significant body of research demonstrates that high-quality teaching is one of the biggest factors impacting student learning and well-being. It can also play a role in improving equity, as several years of outstanding teaching may in fact offset learning deficits of disadvantaged students (Béteille & Evans, 2019). Investing in effective school leadership is as important. Research shows that effective support from superiors makes a substantial difference in teachers’ ability to improve learning. Schools – especially those serving the most disadvantaged – do not tend to improve pupil /student outcomes in the absence of skilled leadership. As it is primarily through the guidance given to teachers that school leaders achieve success, better leadership practices to make optimal use of current teachers can be a low-cost strategy for improving learning outcomes (Bloom et al., 2014; Grissom et al., 2021; Leithwood et al., 2004).

That is why, across countries and regardless of the education sub-sector prioritized together with our government partners, VVOB focuses on introducing and nurturing effective teaching and leading practices and on ensuring that in-service and future teachers and school leaders have the necessary competencies to apply them. In Rwanda, for instance, the focus of DGD programme 22-26 is on strengthening the capacity of school leadership to use data to support teaching, and adequately address repetition, drop-out and equity gaps in learning outcomes.

There is a marked gap between what evidence suggests makes for effective teacher professional development (TPD) and the realities of many TPD programmes around the world (Popova et al., 2019). A compounding challenge is that many characteristics of effective professional development that lead to impact at a small-scale can present real challenges when implemented at large scale, because they are too costly or require high-quality trainers, coaches and facilitators that are simply not present in the education system. We know from experience that the same is true for school leadership development. In fact, many countries have yet to establish professional development (PD) systems for school leaders or they are only in the early stages of doing so.

VVOB’s unique contribution is to accompany governments on this journey. Together with Ministries of Education and institutions for pre-service education and training, induction, and continuous professional development (CPD) of teachers and school leaders, VVOB develops, pilots, tests and scales professional development programmes that fit within the existing system and equip teachers and school leaders with the competencies they need to apply effective teaching and leading practices. To facilitate scaling, we invest in research to increase the credibility of these programmes and in policy dialogue to strengthen regulatory frameworks and mobilise government (financial and human) resources.   

In Rwanda, VVOB’s “Learning through Assessment and Data (LEAD)” programme aims to support SDG4 by ensuring that all Rwandan children can actively participate in quality education and successfully progress through primary education.(see figure 1 - high level Theory of Change)

With the objective to improve the quality of teaching, in particular, with regards to addressing repetition, drop-out and equity gaps in student learning outcomes, (pathway 1), we continue to work with headteachers, deputy headteachers and School-Based Mentors (SBMs), who we identify as the main school leaders. As the use of data by these school leaders is currently poor and actors need to work together effectively, the focus at this level lies on improving the competencies of newly assigned school leaders to using data to support teaching and apply effective distributed school leadership. We believe that an effective school leader plays an important role in creating the appropriate conditions for teaching and learning. Reason being that by supporting, evaluating and developing teachers’ capacity it does not only contribute to the retention of qualified and motivated teachers (SDG 8) but also by doing so improves students learning outcomes, and equity gaps (SDG 5) such as gender (Desimone, 2009) (pathway 2a and 2b). We focus on newly assigned school leaders, as we expect that this is where we can achieve the biggest gain. Acknowledging that newly assigned headteachers, deputy headteachers and SBMs are part of a larger system that can enable or hamper their efforts, especially when they are new, district and sector education officials are also involved and are supporting effective distributed school leadership in primary schools and stimulate peer learning (pathway 3).

To ensure headteachers, deputy headteachers, SBMs, district and sector officials have such competencies, we continue to deepen our long-lasting and fruitful partnership with REB and the UR-CE. Together with REB, NESA and the UR-CE, existing Continuous Professional Development (CPD) modalities (CPD programmes and peer learning networks) are being redeveloped so that head teachers and deputy head teachers gain competencies to lead their schools using data (pathway 4a), and SBMs (pathway 4b) and district and sector officials (pathway 5) gain competencies to coach, mentor and stimulate peer learning of teachers and school leaders respectively.

Concurrently, we work together with MINEDUC, REB and NESA to build a stronger enabling environment for institutionalizing and scaling of these CPD modalities (pathway 6). This supports the further institutionalization of professional development for school leaders in Rwanda. In this regard we  work hand in hand with REB and NESA to ensure that the professional development modalities are recognized as mandatory for newly assigned school leaders. To also stimulate better use of data by REB, NESA and MINEDUC in planning and decision-making, we work together with our partners to establish planning, monitoring, evaluation and accountability mechanisms for the delivery of CPD programmes at central, district and sector level.

How are you currently using evidence within your theory of change?

The Theory of Change forms the starting point for the development of the monitoring system through indicators, for which data is being collected through ongoing monitoring activities and dedicated research studies.

At the same time, outcomes of monitoring and research studies are also used to inform the Theory of Change. The Theory of Change is used as a living instrument and will be reviewed against changing context and insights from monitoring and research in year 2 and 4 of the programme, as part of an internal reflection and learning process.  

The Theory of Change also forms an important instrument for the external evaluation (year 3 and 5). It will be used to inform the Terms of Reference and more specifically to help identifying key evaluation questions and gaps in available data. Evaluators will also be asked to review and redraw the Theory of Change, based on their understanding of how the programme works (as part of the inception report) and on the findings of the evaluation (as part of the final evaluation report).

The LEAD programme is evidence informed as it puts in practice and promotes a school leadership system based on insights from international research findings on the topic of school leadership and capacity development of school leaders. Next to that, the LEAD programme builds on interventions and VVOB support programmes in basic education in Rwanda and on insights and learnings gathered from research and internal and external evaluation studies.

Within the LEAD programme, opportunities are being created to also generate new evidence by monitoring and evaluation of effects and impact, and to learn in iterative learning cycles. A focus area for these learning cycles is the evaluation of the cost-effectiveness of these CPDs and a re-development of the CPDs for delivery at a lower cost. A broader research agenda is being developed in consultation with the programme partners, with following potential areas of research:

• Qualitative research to further explore blended delivery of CPDs.
• Quantitative research to further explore data collected through the digital M&E data ecosystem
• Midline study to assess use of data in PLCs, teaching and leading and changes in distributed leadership attitudes and practices
• Endline study focusing specifically on changes at the level of teachers and students

When applying the Nesta’s Standards of Evidence to the LEAD programme and current practices, we position ourselves at level 2. We gather data which evidence changes at the level of beneficiaries involved in the programme, but cannot yet evidence a causal link. There are however efforts to reach level 3 (e.g. with the study of Maastricht University, see further) and the ambition of VVOB as an organisation is to grow into level 3 in the coming 5 years. Research has been included more prominently in VVOB’s generic theory of change.

How are you currently tracking and measuring your solution’s impact?

In the previous programme, an M&E system was developed to monitor and evaluate the effects and impact of CPDs on school leadership at the level of the trainees and at the level of the schools. Following the Kirkpatrick training evaluation model (Kirkpatrick, 1994), data is collected regarding the reaction of trainees (satisfaction), their learning and behaviour change. At the level of the schools, it is measured whether these training have an impact on the teaching and learning culture in schools. VVOB has worked closely with a research partner (Laterite) to develop a digital data ecosystem which has largely automated the monitoring and evaluation of CPD programmes. (see figure 2 - digital data ecosystem)

Figure 2: printscreen of VVOB Rwanda's digital data ecosystem overall summary

For the LEAD programme, the main indicators are presented in the following table:

The relationship between school leadership and learning outcomes is a point of interest in our research agenda and is currently not reflected in the indicators we follow up. A recent initiative was taken to assess the direct impact of school leadership from VVOB with student outcomes in partnership with the University of Maastricht, where the results were found to be compelling but incomplete (Haelermans and Paul, 2021). Connecting the outcomes from school leadership programming to changes in behaviour at a school level and then to changes in classroom teaching requires very complex models and data available at multiple levels. Currently, VVOB Rwanda has developed a digital data ecosystem that links school leader CPD participation to learning, behavior and school level changes. It still lacks the link to improved teaching and student outcomes.

One-line project summary:

Strengthening school leaders to effectively use data to support teaching, and address repetition, drop-out and equity gaps in learning.

What is your solution’s stage of development?

Pitch your LEAP project: How and where would integrating evidence (or stronger evidence) into your theory of change increase your organization’s impact?

With the support of the LEAP fellowship programme, we see a crucial opportunity to grow in the research line connecting school leadership with student outcomes at the start of our LEAD programme. Here we particularly would like to focus on the LEAD programme’s influences on distributed school leadership through effective use of data on student short term and long(er) term outcomes. We think that these influences can be an appropriate point of departure for a research design that allows us to measure the impact of distributed school leadership on student outcomes, and the role that continuous professional development can play to strengthen this. However, we are unsure how exactly this all fits/connects in one research design. A desk review has been drafted which points out primarily conceptual and methodological issues that may affect studies of the impact of school leadership training on learner outcomes.

With the LEAD programme having a runtime of 5 years up to 2026, the coming months will be crucial to set up a research design, to formulate appropriate indicators and to develop data collection tools. We see a clear role for the LEAP Fellow to support our M&E and Research team in Rwanda. As of 2023, new cohorts of (newly assigned) school leaders will start their professional development trajectories, offering an opportunity for longitudinal data collection.

Evidencing the connection between school leader professional development and learning outcomes has also been a returning request of our government partners in Rwanda, in view of the return on investments. Next to this, there is a considerable amount of data collected by government agencies, which might be very useful in our research design to account for school level and non-school level factors affecting student outcomes.