Seenaryo

Who (first and last name) is the Team Lead for your application and LEAP Project?

Sara Ghanma

Describe the role the Team Lead plays in your organization. [100-200 words recommended]

Sara Ghanma is the Head of Education and oversees (technically and strategically) Seenaryo’s Education and in-schools programming. Sara has been working as an Education Manager on the ‘Seenaryo Playkit’ programming, our play-based training and resources for teachers, over the last 4 years. Currently Sara provides oversight on the resource and training for the Seenaryo Playkit as well as project managing and grant managing the ‘Scenechangers’ programme which is a new in-schools programme that involves training youth to run play-based sessions to deliver the curriculum in schools. Sara manages both the Education Managers in Lebanon and Jordan as well as the Scenechangers Project Manager in Lebanon. Sara has experience in the technical element of designing training content as well as an ability to  support on the technical elements of app development. Sara also has strong coordination skills and she is currently developing her expertise in monitoring and evaluation.

Explain how your Team Lead and supporting team members are well-positioned to effectively support the LEAP Project, given other priorities within your organization. [200-500 words - recommended]

The Education Team consists of the Education Lead based in Jordan (Head of Education), one Senior Education Manager in Jordan and one Playkit Manager in Lebanon. We also envisage hiring a junior Education Manager in Jordan, later this year. The team is supported by two Executive Directors (one based in Jordan and one based in London, UK) as well as by the CEO and Founder of Seenaryo who is also based in London but travels regionally. 

The Head of Education (Sara Ghanma) will lead this project and be the main contact during the sprints, closely supported by the Executive Director also based in Jordan. Together these are the two individuals in the organisation who have the most experience working on the content of the Playkit training and on the monitoring and evaluation of the Playkit.

The rest of the Education Team are located in Jordan and Lebanon so they will be able to implement and support the research needed in both countries. The Education Team has conducted multiple research projects during their work at Seenaryo and previous positions and so are qualified to successfully take on the demands of the LEAP project. 

Seenaryo is a small but growing organisation and therefore the Team Lead is also the front line implementer of the project. Our team are used to playing multiple roles - coordinating project delivery as well as focussing on the development of the project in all its aspects including the training content and training plans, the resources including app development and monitoring and evaluation. To date our MEAL system has been updated with the support of an external consultancy that took place 2019-2020. However, all of the implementation and data analysis and collation since has been in-hosue, giving the team a hands-on opportunity to enhance their skills and experience in evaluation systems and tools.

One-line solution summary: In 20 words or less, summarize your organization's product or program that is the focus of your LEAP Project.

Seenaryo’s Playkit: mobile app and training that up-skills teachers in play-based, participatory teaching, to deliver curricula to 3-8 year olds.

Define the problem that your solution seeks to solve. [200-500 words recommended]

The overlapping crises in Lebanon and Jordan, on top of hosting the highest per capita proportion of refugees in the world, have left education systems in both countries overstretched, underfunded and at breaking point. Both the Lebanese and Jordanian governments identify their biggest problem as the ‘under qualified and unskilled teaching force’ (RACE II). This worsens at early years level, where teaching is often seen as unskilled childcare. This is despite the fact that under 5s are at the most vulnerable while having the capacity to become ‘resilient for life’ (Harvard Centre on the Developing Child).

In 2021, the Ministry of Education (MoE) in Jordan announced that 130,000 students transferred from private to public schools putting enormous pressure on an already stretched public school system. In contexts as strained as these, student drop-out rates are at-risk of increasing and child labour, in the poorest areas, can escalate. 

ECE is an opportunity to intervene before these issues take hold, as well as to combat the major barrier, the low value ascribed to education, by creating value early on. Ensuring that ‘pre- and in-service professional development meets the needs of educators and learners’ is an urgent priority according to the National Statement of Commitment-Jordan and providing ‘at least one year of equitable, high quality and inclusive early childhood education for all children a year before the official primary entry year’ is a key priority in Lebanon’s 2022-2025 General Education Plan. The investment in the Playkit as a solution that seeks to enhance and support the efforts of the Ministries of Education in both countries.

Describe your solution and how it works in simple terms. [200-500 words recommended]

The Seenaryo Playkit is a mobile phone application and training that uses play to revolutionize early childhood education. It is intended for teachers working with 3 to 8-year-olds in under-resourced schools in Jordan and Lebanon. The Playkit provides an extensive collection of interactive stories, games, songs, and classroom management strategies, all of which can be easily searched by topic, age group, or activity type, and can be adapted to suit any early childhood curriculum. The app includes instructional videos, music tracks, and printable flashcards in Arabic, English, and French. The Playkit is typically accompanied by a four days teacher training, and aims to foster children's cognitive and language, social and emotional, and physical development by utilizing participatory, child-centered, and inclusive approaches.

The Playkit content is designed to contribute to building life skills among children across 5 developmental areas of the brain: social, emotional, physical, cognitive and language. Specifically, the Playkit targets 7 life skills within these area including communication, self-regulation, cooperation, resilience, fine & gross motor skills, decision making and problem solving. The idea is, that before children can achieve the academic objectives that teachers and curriculum set out for them, they need these life skills to be able to engage in their learning and thrive in their classrooms. Structuring activities with a life skills focus allows teachers to include children of different abilities and learning styles, for instance through the use of non-verbal participation (songs and games with actions). Teachers learn to provide a a range of options that children can use to respond to questions or to participate in an activity, moving away from the traditional classroom structure of a teacher speaking and children listening.

Select the key characteristics of your target population. Select all that apply.

• Women & Girls
• Pre-primary age children (ages 2-5)
• Primary school children (ages 5-12)
• Poor
• Low-Income
• Refugees & Internally Displaced Persons
• Minorities & Previously Excluded Populations
• Other

If you selected “other” please indicate which populations you serve.

Specifically - teachers of children aged 3-8 years old in under-served areas in Jordan and Lebanon. By under-served we mean populations of refugees in both countries, as well as native populations facing poverty, social exclusion or other kinds of marginalisation. 

In which country or countries does your solution currently operate?

Jordan, Lebanon (Support office in London, UK)

Upload your solution's Theory of Change or Logic Model.

Where would you place your solution on Nesta's Standards of Evidence?

To date, what research/studies has your organization conducted that have helped demonstrate the effectiveness of your solution? [200-500 words recommended]

In 2017, Seenaryo was commissioned by one of our funders to investigate how our existing theatre tools currently being used in communities could be adapted to serve the needs of teachers in schools and in particular to address a gap in teacher training and resources for KG teachers. At that point we began to conduct formative research, we collaborated with teachers from the International Rescue Committee, Ana Aqra (a local Educational CBO), and Caritas in Lebanon to carry out a needs assessment and conduct some research and development. This involved talking to teachers to understand what they were currently using and what they felt was missing as well as trialling activities in their classrooms and reviewing and revising the effectiveness of these activities with the teachers themselves. 

As we rolled out the Playkit over the course of the first couple of years in schools in Lebanon and Jordan and continued to adapt activities and training plans based on teacher feedback, we also collated the evidence and research behind the theory of this approach into the Playkit Background Document. This document briefly outlines the history of play and its use in education and details the rationale for addressing specific life skills and how this fits into the picture of overall childhood development, from a neurological perspective. This involved desk research, literature reviews and consultation with experts therefore constituting our foundational research. 

Our monitoring and evaluation system and corresponding set of tools is an ongoing process of formative research. We use teacher surveys, focus group discussions, headteacher interviews and classroom observations to assess the impact of the Playkit training and app on teacher behaviour change. Seenaryo’s MEAL system has been supported by a consultancy with a MEAL specialised company in Jordan (Integrated) who worked with us on developing our Theory of Change as well as the corresponding indicators and tools. In addition we have been involved in research training projects such as ‘Saving Brains’ - this was a five day workshop with other innovator teams across the world where we were able to share our solution, evaluation tools and approaches with others in order to get their feedback. The workshops focused on developing the workplan Seenaryo needed in order to scale the Playkit. 

Finally, the only summative research we have conducted was an informal study in order to compare the effectiveness of a 1 day training compared with the standard 3 day training. We ran several 1-day training sessions and compared the evaluation results (surveys, focus groups and classroom observations) with the results from 3 day training. The findings of this research was collated into a report and helped to inform our strategy for the Playkit moving forward.

What has the research/studies you have conducted revealed about your solution and how did it inform your work moving forward? [200-500 words recommended]

The findings of the initial Needs Assessment and R&D process in 2017 were aligned with the Crisis Response Plans of the Lebanon and Jordan governments, indicating that the educational delivery was not learner-centerd and lacked a life skills-based approach. At this point we had the information we needed to develop the first draft of the Seenaryo Playkit book of activities and initial teacher training plans. 

The results of Seenaryo’s MEAL system have been recorded throughout the years in reports  specifically for donors or as part of Seenaryo’s annual Impact Report. More recently all data from Playkit M&E has been combined into a single report mapping overall impact of the Playkit since its inception in 2017. As of April 2023, Seenaryo has trained 2,643 teachers, reaching 67,799 children aged 3-8 years old. In terms of outputs, our M&E has helped us gather data that indicate progress against the Playkit objectives. For instance we can see overall that there has been a shift in teachers’ attitudes towards play-based learning; before taking the training 51% of teachers disagreed or strongly disagreed that ‘children learn best when sitting at a desk’, compared with 77% post training, suggesting that the Playkit significantly increases teachers’ understanding of the value of participation and play in the classroom. We were also able to gain feedback from this evaluation style research on the value of the training and the value of the Playkit content, 96% of more than 1000 teachers surveyed agreed or strongly agreed with the statement,  ‘I feel the Playkit resources helped improve my teaching practices to positively manage the classroom’. Other significant data includes evidence from Classroom Observations where teachers have demonstrated improved ability to positively manage behaviour, 72% of teachers observed appears in control of their class post-training compared to 30% pre-training. 

Seenaryo’s evaluation over five years has demonstrated the Playkit’s significant, positive impact in supporting teachers in all regions of Lebanon and Jordan to understand and implement participatory, play-based teaching. The Playkit’s impact has been particularly pronounced and well-documented in effecting positive management of classrooms. Combined with the results from the quick fire research into the effectiveness of 1-day trainings, we have been able to make several evidence based decisions about the future. We know that we need to invest in the evaluation  of the Playkit, particularly in terms of measuring impact on children. Secondly, the evidence from the 1-day trainings suggest we need to focus on the Playkit as a resource and how to support the use of that resource, beyond training. The 1-day training was successful in helping teachers gain a base level of skill but to further support that roll out, at scale, we need to invest in the functionality of the Playkit app and in the concept of ‘teacher communities’ - an online and in person space where teachers can get coaching and mentoring from Seenaryo, as well as gain peer support and exchange best practice with other teachers.

Describe your organization's need to strengthen the evidence base of your solution. [200-500 words recommended]

Seenaryo has reached a point in its journey where we have robust foundational evidence for the impact of the Playkit, in the 4 day training form that we typically use. However we have identified two crucial elements that are missing from this evidence base in order to move to the next step of scale and reach that we envision. Firstly, we have not been successfully able to measure the impact of the Playkit on children. As a small organisation with limited capacity we were originally advised and felt it realistic to focus on our specific sphere of influence i.e. teacher behaviour change and so our Monitoring and Evaluation tools have focused on that as the key measurement. However, now that we are aiming to scale and have the opportunity to invest in Playkit Resource (i.e. the app), we want to ensure that we do so informed not only by teachers voices but also by the children's voices, the impact on them and the learnings from their experiences of being taught by ‘Playkit’ teachers. 

The second area we identify as needing support at this crossroads moment of Seenaryo Playkit development is being able to support the evidence we do have with research from the field and therefore proving the causal links between teacher behaviour change, increase in life skills and improved life chances for children more effectively. This is part of being able to analyse and distribute the evidence we do generate through our monitoring and evaluation systems effectively. This will allow us to effectively advocate not only to institutions and ministries within our countries of operation but will also allow us to convince institutional donors of the impact and success of the Playkit, leading to increased funding and therefore a greater ability to scale and reach many more teachers and children. 

After an initial period of growth and fast development during the first few years of the Playkit, Seenaryo has spent the last 12 months reflecting on programme design, building out our internal systems and processes and refining the quality of what we do and how we do it. Now we are in a position to tackle growth and development again and part of this is upgrading our evidence collection or monitoring and evaluation systems. With funding confirmed from TheirWorld and Kindernothilfe within the last few months, Seenaryo has the funds to support the salaries of the Education Team who will be working on the LEAP project. In addition to Head of Education (Sara Ghanma) and Education Manager (Carole) we will be employing another Education Manager in order to free up some of their time and ensure there is capacity for participation in the LEAP project.

What are 2-3 research questions that you would like your LEAP Project to help you answer? [100 words recommended]

• What is the most cost effective and efficient way (with capacity limitations in mind) to include children in our evaluation, with a focus on participatory evaluation? This is with a view to understanding the impact of the Playkit use in classroom, from the perspective of the students. 

• What are the best systems for analysing, using and distributing the evidence that we are currently generating from our Monitoring & Evaluation systems, in order to advocate locally to stakeholders and externally to donors and institutional funders?

What type of research/studies do you think will help answer your stated questions? Select all that apply.

• Formative research (e.g. usability studies; feasibility studies; case studies; user interviews; implementation studies; pre-post or multi-measure research; correlational studies)
• Summative research (e.g. correlational studies; quasi-experimental studies; randomized control studies)

Please elaborate on your selection above by describing your desired outputs of the 12-week LEAP Project sprint. [200 - 500 words recommended]

Over the course of the 12 week project, we are hoping that our team to develop greater understanding and knowledge in the following areas:

• A better understanding of how to conduct effective participatory evaluation with children 

• Greater knowledge about the ethical use of control groups when measuring impact of children and teacher interventions

• Insight into a range of monitoring and evaluation databases that work to analyse data and generate reports with the hope of discussing the pros and cons and functionality of these systems with other LEAP fellows

• How to better use different types of evidence for different audiences and how to present evidence of the impact of the Playkit in a way that highlights its strengths most effectively, particularly for key local stakeholders (e.g. ministries of education) and donors

In turn we hope this knowledge and understanding will help us to produce, within the 12 weeks:

• A range of tools that can be used for children’s participatory evaluation, and a chance to pilot and feedback on these tools within the 12 weeks

• A decision about whether or not to use control groups and how to do this, producing a guidance document on control groups if we do decide to do this

• Decisions and a roadmap for a system and accompanying software that we will adopt for data analysis and report generation. We hope that once we’ve made this decision that at least 1 or 2 of our LEAP fellows will gain skills in using this system and be able to train the rest of the team

• A roadmap for reaching stakeholders and donors using our evidence base that can feed into our Playkit strategy

How will your organization put these outputs into action? [200-500 words recommended]

If we are successful in achieving the outputs listed above, we hope that our organisation will immediately be able to integrate the new understanding and tools into current systems and processes in the implementation of the Playkit. This starts with a new approach to children’s participatory evaluation which we will implement both pre-training and post-training. The children’s participatory tools we develop will also be integrated into our overall MEAL system, adjusting indicators, workflows and Theory of Change as needed. Another key action around the new evaluation tools for children will be training some of our freelance facilitators to lead these sessions with children. This may not be possible during the 12 week period and therefore this will need to be one of the first actions we undertake after the LEAP project.

Seenaryo is also currently engaging with University College London on a longer term research and evaluation project in which expert researchers from UCL in this field are conducting a broad review of all of Seenaryo’s Playkit implementation and MEAL tools. Therefore we hope to integrate our findings from the LEAP project into this larger scale research and adopt the learnings into other tools (i.e. tools for teachers) where appropriate. We hope that the combination of understanding and insight we gain from the LEAP project, as well as the research with UCL will help us to secure a strong foundation in evidence gathering and position Seenaryo to strengthen its impact and influence across Lebanon, Jordan and eventually, across the whole region.  

The second key action will be, over a period of time, adopting the new system and/or software for analysing data and integrating all of our MEAL tools within this system. We predict that this will also require 1-2 months of building out the system and training staff on its use. We hope to use it to effectively store and analyse data, as well as to generate specific information and analysis that we can use to report relevant information and impact to different audiences.

Describe your desired short-term and long-term outcomes of the 12-week LEAP Project sprint for both your organization and solution. [200-500 words recommended]

Seenaryo envisages the following short-term outcomes after the 12-week LEAP project sprint:

• A more robust system for anaylsing data and generating reports resulting in Seenaryo’s ability to better showcase Playkit evidence, tailored to different audiences and linked with research in the field

• With a sound and renewed evidence based, we hope to gain more exposure and buy-in with stakeholders such as Ministries of Education and larger INGOs operating in the field (e.g. Queen Rania Teaching Academy)

• Greater access to government schools, through the Ministries of Education

• A stronger profile in the region attracting interest in collaboration projects and access to schools through other INGOs

• Greater exposure to and convincing evidence for donors and institutional funders, leading to the ability to secure larger grants and therefore expand the reach, scale and quality of the Playkit through investment in the app, staff costs and a wider geographical implementation

In terms of long term outcomes we hope that Seenaryo, through the Playkit solution, will be able to reach hundreds and thousands more children aged 3-8 in Lebanon, Jordan and at least one other country of operation by the end of 2024. We will reach them through more effectively training their teachers and more effectively garnering the support of stakeholders. If we achieve that, the long term outcomes we envisage for the Playkit and the thousands of teachers and children we reach are that: teachers are more fulfilled and inspired; children learn in an environment that is inclusive, child centred and engaging and children develop the life skills they need to reach their potential. Ultimately we hope this leads to the systemic change in Lebanon and Jordan, that individuals become powerful agents of change & contribute to stronger, more vibrant communities.