Nudges to Increase Routine Immunization

In 2020, 23 million children failed to receive their basic vaccinations (WHO, 2021). Vaccination coverage varies widely between and within countries, so certain regions bear a much greater load than others when it comes to vaccine-preventable diseases. India has around half of these world’s undervaccinated children and suffers the death of one child every four minutes as a result of a vaccine-preventable disease (WHO, 2020). Around 10 million children born in India each year do not receive their full recommended set of vaccinations by their first birthday. Of these undervaccinated children, over 7 million receive their first basic vaccinations, but fail to complete the rest of the course by the recommended age. 

This “drop-off” is the key problem that Suvita aims to address to improve immunisation coverage. Analysis indicates that root causes of this drop-off in India are primarily demand-side challenges (where parents do not access the services available to them) rather than supply-side challenges (where services are unavailable). For example, a survey of nearly 40,000 parents in low-immunisation districts found that the most common reason for missing a vaccination appointment is an awareness gap. Other analysis has identified that caregivers also commonly drop-off because they forgot to attend their appointments, or because they are worried about or inconvenienced by side-effects (if the baby gets a fever, it might stop the caregiver from working or strain family relationships). In a nationwide survey, caregivers were four times more likely to identify demand-side barriers than supply-side ones when explaining why their child had missed a vaccine dose. Furthermore, roughly 70% of undervaccinated children in India have received at least one vaccine, indicating that missed appointments are a major barrier to full vaccination. Banerjee and others note in their study in India that while vaccines are free and available, a combination of relative indifference, inertia, and procrastination results in children dropping out. 

With the current pandemic, this number of drop-offs has increased. A side effect of India’s tough stance on COVID-19 has been that millions more children are missing out on vital vaccines they should receive in the first year of their life. The Indian government’s expansion of immunisation since 1990 cut the deaths in India from these diseases by 450,000 every year. As a result of COVID-19, we are at risk of losing that progress. Without targeted solutions, disruption of routine vaccination across India could lead to new outbreaks and new epidemics in a few short years.

Our work is focused on India which has 7 million children not completing their routine vaccination schedule every year. Suvita implements two rigorously-tested models to increase uptake of vaccinations and protects infants and communities from future pandemics. Our work is based on research by Nobel-Prize-winning economist Abhijit Banerjee and colleagues. In their high quality RCT conducted in Haryana, India, these economists found that these two programmes when implemented together increased immunisation rates by about 20%: 

1. Sending personalised SMS reminders to parents in their local language every time their child is due for a vaccination.  They encourage mothers to take the first, critical step of talking to their ASHA, a government appointed community health worker,  or going to see a nurse when it is time for their children to be vaccinated. In this way, reminders can directly reach mothers with critical health information, even where the ASHA is unable to. While our personalised SMSes emphasise and repeat that vaccination prevents deadly diseases, our automatic classification and data extraction process for birth record documents ensures that the immunisation SMS reminder program can be scaled up to different cities  without the need of onboarding more data entry personnel.

2. Building a network of carefully selected volunteer immunisation ambassadors, who share information with new parents in their community about local vaccine clinics and schedules. In each community where we work, we establish a network of immunisation ambassadors who are nominated by their community members for their ability to spread information. We chose a remote rather than in-person survey approach to identifying these ambassadors because we expect it to be more scalable and cost-effective, and it has also proven highly adaptable to the current pandemic. These local ambassadors are regularly motivated (using SMS messages and phone calls) to spread information to caregivers about the importance of getting their children vaccinated, using whatever approach they believe to be most appropriate for their own community. They then mobilise caregivers to bring infants to routine immunisation sessions. 

Ambassadors apply their knowledge of the local context to choose a targeted strategy to address local barriers to vaccine uptake, while SMS reminders provide a behavioural nudge at the optimum moment. This flexibility combined with personalisation and the use of  advanced technology such as AI and machine learning enables the programmes to meet complex and varied needs at scale.

Our end users are the parents of the 7 million children every year in India who start but do not complete their vaccination schedule. We work primarily in Bihar and Maharashtra states, where over 1 in 4 children do not complete their full vaccination schedule (NFHS-5, 2021). Studies indicate that these families are more likely to have certain demographic characteristics such as having low income and low education levels; being less well-connected to the health system in general (e.g. having had fewer or no antenatal care appointments, and being more likely to have given birth at home rather than in hospital); and belonging to minority religions and historically oppressed castes and tribes. 

We design our programmes to be accessible to our users by sending all messages in local languages using the ubiquitously available technology of SMS messages. Whereas almost every family in India has access to a mobile phone, only around ⅔ of people have access to smartphones. This means that SMSs can reach many of our users where e.g. WhatsApp could not. We are also currently exploring the alternative of recorded voice messages in addition to SMSs, as these are similarly accessible on basic phones, while also overcoming the literacy barrier faced by communities with fewer years of education. Families also do not need to spend any mobile credit to enrol for the programme (this is carried out through either automatic enrolment when they give birth at a healthcare facility, or via a missed call mechanism that costs them no money). 

We identify natural influencers in a community network through the magic question: “If there was a fair in town, who is most likely to tell you about it.” The people who are nominated tend to be active and well-connected in their community, meaning that they have a much better idea of key barriers to immunisation within their own networks, so they can take whatever approach they expect will be most effective in their local area. We expect that this also means the ambassador programme is robust to variation in specific needs and barriers if delivered at scale, because in any community it is led by those who know the community best.

In India, one child dies every 4 minutes from a vaccine preventable disease. This is unacceptable. I grew up in India and am a mother of two young children. I can relate to the sense of being exhausted, disorganised and overwhelmed with infants and can intuitively see the value of such nudges in improving uptake of routine immunisation. 

We are a sharp and agile team of 28 generalists and experts committed to solve the problem for good through a relentless focus on impact and scale and a culture of constant iteration and learning. We see this approach as a highly promising route to impact - taking programmes that have been shown to work and helping them reach as many families as possible. Our leadership team spent thousands of hours reviewing the literature on the most cost-effective and evidence-based global health interventions to select and refine our focus. We are confident in our solution and relentlessly dedicated to saving and improving as many lives as we can. 

Our solution depends on the people in the communities we serve. We talk with them to find quality immunisation ambassadors, whose feedback allows us to refine our approach based on personal experiences and deep community ties.

In Saran district, Bihar, where we undertook our initial study this year, we conducted Receipt & Understanding surveys with SMS enrollees and interviews with immunisation ambassadors to better understand their motivation and impact. We are also tracking parents’ and caregivers’ responses to ambassadors by giving out a phone number to ambassadors to share with their networks. Our monitoring suggests that more than 98% of text messages are delivered successfully and at least 50% of the users recall the information at a later date. Overwhelmingly ambassadors are enthusiastic and sharing information with their networks, we received about 30 missed calls within one week of sharing a missed call phone number with 45 ambassadors. 

We also use qualitative research methods such as interviews routinely to get a deeper and better understanding of the program’s benefits (and problems) on ground. For example, one of the interviews with a parent we work with, Sonu, in Bihar state, India stated: “This is your very good initiative, everyone who receives this SMS should go to vaccination. Whatever your organisation is doing is commendable - keep doing this - this will secure the future and health of many children.” 

I do not know of any other similar opportunity to engage with entrepreneurs, investors, governments, and academics, all focused on sharing ideas and building relationships to increase social impact.  I look to harness the collective and individual wisdom of colleagues to inform our strategy on approaching scaling of our operations in India and abroad. We at Suvita have been in the startup ideation mode for the last two years and believe this year is pivotal in order to chart our exponential scale up plans. Specifically, helping me think through different models of scale, how to think through evaluation of our programming, and how to focus more on external communication would be crucial for us to reach our goals this year. 

As members of the Solve community, we would like to work with experts in technology, data security, monitoring and evaluation, scaling, and the implementation of immunisation ambassador programmes in India (an area in which MIT’s J-Pal team has unique experience). We also hope that a Solve grant would help us reach our short-term budget targets and introduce us to partners who could strengthen our long-term financial outlook, either through direct funding, coaching or connections with other funders and government partners.

Organizations and individuals experienced in maximizing the cost-efficiency of public health outreach programmes could help us reduce our cost per fully immunized child from $64 to $36, and identify other opportunities for increased efficiency.

Financial: Since all of our goals are dependent on funding and we are still in the pilot stage, we stand to gain more from financial coaching and assistance than we do in any other area. Fundraising support could include introductions to investors (especially those with some connection to health efforts in India) and preparation for investment discussions. We would also benefit from  coaching to build stronger financial and accounting systems, and strategic planning to build a sustainable long-term funding model. 

Monitoring and evaluation: We would like to increase operational efficiency, measure and demonstrate our solution’s impact, and drill down on the  mechanisms that most contribute to its success. For example, we would like to partner with third party evaluators to learn more about how our general SMS reminders and immunisation ambassador nudges interact to make better use of this unique model. This would help us answer questions like, “To what extent are ambassadors reaching parents and caregivers that our general messages don’t, and how does that impact immunisation rates in each village?"

We believe we are the only organisation using an evidence-based immunisation ambassador approach in addition to general mobile health reminders. Several NGOs and government bodies include immunisation reminders as a component of wider programmes, but each of these is limited by at least one of the following:

• A small target geography and a model with limited scalability.

• Inability to reach most of their target population.

• A narrow target population of wealthier users.

Our nudges are based on research at the forefront of the immunisation sector - a groundbreaking study, published just this year. In particular, recruiting deliberately-identified network-central individuals as Immunisation Ambassadors is a novel approach for increasing routine immunisation uptake. We have further innovated on this work by designing a more scalable (remote) model, which will enable us to support more children, more quickly. We conduct our own internal research and innovation in our continuous improvement ‘sandbox’, whilst also collaborating with the original research organisation, J-PAL, to contribute our own learnings to their ongoing research.

More broadly, the idea of starting a nonprofit specifically to spin out from high-quality research findings to deliver an evidence-backed intervention at scale is an innovative one, having only been adopted by a handful of nonprofits worldwide to date.

Finally, our solution is innovative because it builds on existing and widely available low-cost technology, and combines it with behavioural science, network theory and artificial intelligence. SMS reminders ensure that caregivers get encouragement and reminders at just the right time, personalised to their context, to bring their child for vaccination appointments. The community ambassador programme builds on social networks to further amplify impact. Since SMS reminders might not reach all families or caregivers, local ambassadors increase the likelihood that a caregiver will bring their child for their routine vaccines by improving access to information (location and timing of local immunisation outreach sessions), and strengthening social norms.

In 1 year, we aim to hit the following targets:

• Enrol 25,000 infants per month into our general SMS reminders by streamlining our SMS delivery process and reaching out to more mothers in Maharashtra. 

• Recruit 50,000 immunisation ambassadors by expanding on the findings from the initial phase of our pilot to expand more efficiently. 

• Move our cost effectiveness for remote immunisation ambassadors from $64 per marginally fully immunised child to $36 (through decreased senior staff cost per child as we scale up, tech innovation, and operational efficiency gains/innovation)

 We plan to substantially strengthen our monitoring & evaluation systems to ensure that we get high-quality feedback to further improve our programmes.  

In 5 years, we aim to have fully rolled out SMS-based nudges across Bihar and Maharashtra. We plan to focus on delivering our models in additional high priority states, such as Uttar Pradesh, based on our learnings around the optimal approach for reaching mega-scale (e.g. whether our model could be effectively implemented by governments and other foundations and non-profits). As we refine our delivery process, we also plan to explore using our model to expand to other countries and address other health issues such as antenatal care. We will have established a robust feedback cycle to identify and deliver programme improvements, and we will also be working towards a large-scale randomised controlled trial to demonstrate our impact. Our best-practice tech and HR systems will support efficient scaling.

We measure our impact by looking at: 1) the number of children enrolled for SMS reminders and 2) the number of immunisation ambassadors recruited (hence the number of children expected to be covered by immunisation ambassadors). We routinely monitor these as part of day-to-day programme delivery. Both of these outputs together drive our two final outcomes, which we plan to focus on to assess our project’s impact: vaccination coverage rates and child mortality rates. 

To connect vaccination rates to child mortality, we use an iterative statistical model developed by nonprofit evaluator GiveWell. This estimates, for example, that a 2.0 percentage point increase in vaccination uptake amongst 10,000 children in Bihar will save 1.92 lives. We also hope that, in future, we will be able to measure changes in child mortality, due to vaccine-preventable diseases, through administrative data.

We currently measure vaccine coverage rates through self-reported caregiver surveys (survey instrument attached), with a random sample of caregivers, designed in such a way as to identify causality (“Do you think your child would have got vaccinated on time even if you had not received the SMS message from Suvita?”) and to minimise surveyor demand bias (e.g. by using primers such as “Some people might find the SMS helpful in reminding for vaccination, but for some people, it may not have been very helpful. Could you tell me...” and “Your honest opinion will help us to improve our program.”): 

We are also currently investigating the possibility of using administrative data to measure changes in vaccination rates. For this, we are participating in a pilot study which uses admin data in Maharashtra and are also preparing to analyse admin data in Bihar.

Caregivers are enrolled for SMS vaccination reminders via data collected from local health centers (in Bihar) or the state birth registrar (in Maharashtra). They receive automated reminders with vaccine-specific information shortly before each of their child’s appointments at government-run vaccination clinics. The content and timing of these messages is shown to inspire parents and children to vaccinate their children. After receiving the messages, parents are more likely to bring their child to the vaccination camps. In 2019, our meta-analysis of nine existing randomised controlled trials confirmed that SMS reminders sent to caregivers increase vaccine uptake by between 3 and 8 percentage points. Increased immunisation camp attendance means that more children are protected from common vaccine-preventable diseases, and others are protected by herd immunity. 

This outcome is enhanced by the work of our immunisation ambassadors, as evidenced by the results of J-PAL’s 2019 study. Randomly selected community members are surveyed to identify local opinion leaders, who we then invite to volunteer as immunisation ambassadors. Those who volunteer are regularly updated with details of local monthly immunisation clinics by automated SMS messages. They share this information with relevant members of their community. (Early surveys from our pilot show that 89% of ambassadors shared the information with family and friends and 67% shared with other acquaintances). Caregivers in the area are then reminded to bring their children to government-run immunisation camps, which further increases the number of vaccinated children. Our ambassador recruitment and deployment process is modelled on the steps used in J-PAL’s 2019 study, which led to a 25% increase in vaccination camp attendance.

To deliver our programme of SMS messaging we use A2P messaging, which is essential to IoT. We use Telerivet and bulk SMS providers Textlocal and HSP. Our head of technology is currently working on a machine learning model for Automatic document classification & data extraction/capture of birth records which will help in scaling up the immunisation SMS reminder program to different cities without the need of onboarding more data entry personnel.

Thoughtful use of technology enables us to get sharp visibility into what is happening on the ground at minimal cost. We use Nexmo Advanced API, a proprietary service, to query the current status of a sample of numbers from each enrollment channel and SurveyCTO a proprietary extension to the open-source OpenDataKit, to implement and conduct our surveys on a tablet or mobile phone. In addition to these, we are also working on a Suvita AI Chat Bot that will use pattern matching along with AI chatbot deep learning, machine learning and natural language processing (NLP). This chat bot will help collect feedback from our users by initiating automated surveys. Additionally, the automated system will also help with common queries and service requests, for example, if someone is worried about the side effects of vaccines, an sms can be sent through the bot which can allay their fears.

We are also working on developing an AI Engine that will help monitor Suvita’s impact and help improve the current performance of the program. This will be done through a statistical model that will help discover the correlation between the different metrics and the vaccine efficiency. Moreover, the engine will leverage a classification algorithm to build parent categories/ profiles to predict specific behaviours and recommend best interventions to counter any vaccine hesitancy behaviour.

Both the co-founders of Suvita are women. 12 (43%)  of our 28 colleagues are females and the rest are males. We ensure our hiring systems encourage diverse candidates to apply and go through the process. For instance, with our current Head of Technology role, we ensured we had gender diversity in the candidate pool, including undertaking statistical tests on the make-up of the candidate pool.We base our candidate selection on evidence and rigorous assessment, rather than personal impressions or interviews. In other words, we go out of our way to ensure our hiring process is merit-based. Beyond regular surveys and focus group discussions, we are exploring ways our end-users can be more involved in Suvita’s decision-making.

We have a clear idea of our core customers- the 7-10 million children who do not complete their routine immunisation schedule every year. More than 70% of these children are in five states in India that are our focus states. Our model relies on grant and philanthropic funding from governments and donors. While we continue to iterate our model to make it more cost effective and demonstrate impact, we are exploring several pathways to scale which involve state or central government and large foundations as doers and funders at scale.

We plan to solve the problem for good over the next ten years. Both our models are highly cost effective. For instance, we believe we can send out SMS reminders to eligible mothers across Maharashtra state for the next three years for approx. USD1.6M. More details available here: https://drive.google.com/file/d/1Jn3cEFaEBwNPf2Ypw7YHlho_qznIJVYz/view

We will build state government capacity to send out relevant, curated text reminders to parents over the next ten years. At the same time, we believe we could set up the cost -effective ambassador model across the top five states in India which have more than 70% of the immunisation drop-off rates to solve the doproff challenge for good over the next ten years. Both these models will require funding for the next decade. We plan to rely on philanthropic funding for both the models. 

Our funding over the past two years has been around USD 200,000 per year funded through a mix of venture philanthropy and Effective Altruism funders. Our funders in 2021 include Azim Premji University, Founders Pledge members, Schmidt Futures, The Funding Network (TFN), Network for Social Change,  Mulago Foundation and Bianca Vetter Foundation.  Funders in 2020 include D-Prize, The/Nudge Foundation , Azim Premji University, Founders Pledge and individual founders connected to them,  Frederick Mulder Foundation, and SITA Foundation.