Immunization MentorChat

Vaccines save lives. But each year, 20 million infants do not receive a full course of even basic vaccines, and of these, over 13 million receive no vaccines. It's often the poorest and the most vulnerable parts of the world that are the least protected by vaccines. And in these areas, healthcare staff often lack the training and skills they need to deliver vaccines safely to their communities. Health systems cannot function without skilled healthcare staff. To enable vaccines to reach more children, healthcare staff need more accessible, personalized, and trustworthy training and support, available to them in the flow of work.

Consider the Democratic Republic of Congo. The country has one of the highest rates of child mortality in the world. In 2017, only 1 in 3 children were fully vaccinated. And 1 in 5 children did not receive any vaccines at all. It is incredibly challenging to provide immunization services to children because DRC has one of the lowest population densities in the world. It is the second largest country in Africa, but it ranks 196th worldwide in population density. Half of DRC’s under-immunized children live in remote, hard-to-reach villages. Some villages are days away from the closest health center. In health centers, there is often a shortage of health workers and a high rate of turnover. As a result, there is a near-constant need to train new staff. Immunization managers may be a few miles away, but they are typically not onsite to provide coaching and mentoring, and they are tasked with overseeing multiple health centers. 

Making matters more challenging is the fact that vaccine programs are complex systems, requiring healthcare staff to manage a tightly controlled cold chain system, administer multiple vaccines with safe techniques, and communicate extremely well to overcome gender-related barriers and vaccine resistance. In-person training is the traditional method, but it can be slow and expensive to implement, especially in hard-to-reach areas. It also takes providers away from their posts. And training is an ongoing need, as vaccine and their practices evolve. 

We launched Immunization Academy, a digital learning platform for healthcare professionals, in 2017 to help solve this problem by offering 200+ short, how-to videos on essential immunization skills. Currently, though, when health workers need help finding an answer, they often post questions to our discussion board rather than using the search tool. "What is the right way to do x?" "Is it OK to give a vaccine dose under y conditions?" The answers always exist, either in our videos themselves or global/country guidance documents. But researching and then providing answers manually, on a discussion board, adds time and effort to the process – which means healthcare workers may not have the help they need, in the moment when they need it. Health systems cannot deliver life-saving vaccines under these conditions. 

Our solution, Immunization MentorChat, uses generative AI to transform the way healthcare workers find the answers they need to assist them in the flow of their work. 

The vision is for healthcare professionals to be able to enter their question in a chat field using natural language and to receive adaptive, personalized answers from a virtual mentor -- answers that have been abstracted from Immunization Academy videos and other credible sources. This brings health workers one step closer to having the immediate coaching and mentoring they need, no matter where they are, from a trusted source –- like their own personal immunization mentor. They don't need to search through a lot of documentation or training, or struggle with a language they aren't comfortable with. They can phrase the question in their own words and receive a short, reliable response -- even in the moments when they're providing service to their community.

Immunization MentorChat is the next step in our evolution as a leading provider of training to the global health community. This generative AI-driven chat tool is offered as an integrated feature of our existing learning platform, Immunization Academy. It leverages ChatGPT and uses the DocsBot AI tool. It is currently in prototype form, while we develop, test and train it. In its more developed form, MentorChat would answer questions with the user's location in mind. If someone asks how and when to give the polio vaccine, MentorChat's answer would be different, depending on whether the user is in Kenya or Zimbabwe. After responding to users with the personalized answer needed, MentorChat would also suggest specific video lessons that users may want to watch to deepen their skills, when they have time. Our next step: to share our prototype with a pool of users to learn more about their wants and needs from the tool. 

Meanwhile, our flagship platform, Immunization Academy, currently offers 200+ short, skills-based, free video lessons to teach health workers and immunization managers all the skills they need to deliver and manage immunization programs. We call it the "YouTube of Immunization." All video lessons are based on globally accepted WHO guidelines, are translated into multiple languages, and have been reviewed and endorsed by WHO's Vaccine Safety Net. Our IA Learn app is an Android app that allows countries to quickly deploy customized training to address their unique performance gaps by selecting from our 200+ short videos, assessments, and real-world practice assignments. Once the app is downloaded onto an Android device, it can be used entirely offline making the training accessible to even the most remote and hard-to-reach health facilities and those in fragile settings. 

Together, these digital resources can help overcome many of the barriers to skill-building -- particularly in low-resource settings. 

Our solution serves health workers at the service delivery level. It impacts their lives by bringing the support and training they need and want right to their fingertips, in a more palatable and personalized format than ever before. 

Immunization MentorChat improves learners’ skills and knowledge, which not only improves their ability to provide high-quality services but is also empowering and motivating. Personalized support is accessible to even the most remote and hard-to-reach health facilities and those in fragile settings. Like one of our users, a health worker in the Democratic Republic of Congo, said, “We don’t need to wait a month for a supervisor to come from Kinshasa and teach us something. We can teach and learn ourselves.”

Our solution also serves health systems by enabling them to rapidly train and support their health workforce. Given the current shortage of health staff (with even higher shortages predicted in the next decade), this digital solution helps fill a very real need to provide healthcare professionals with basic skills. In addition, in times of disease outbreak and conflict, it is difficult for a health system to bring people together for face-to-face training; in those situations, managers are unable to travel to clinics to provide support, because of security or geographical challenges.

Ultimately, a skilled and motivated immunization workforce impacts the communities they serve by reaching children and individuals with the vaccines they need to prevent disease and death.  

Over 900,000 health workers in low- and middle-income countries around the world turn to the educational videos provided by the Immunization Academy (IA) for guidance on how best to administer their immunization programs to mothers and children. On average, our videos get 1,500 views every day. We are already known and trusted by many healthcare workers.

We are not geographically close to the low- and middle-income countries we serve, however we typically partner with in-country organizations (such as the Clinton Health Access Initiative, the Task Force for Global Health, CDC, and Ministries of Health) when designing and implementing learning solutions. BCL is an award-winning learning consultancy based in Durham, North Carolina, USA. BCL has been assisting global health organizations in building workforce capacity through the creation and delivery of innovative learning solutions for over nine years. In that time, we have produced hundreds of digital and face-to-face learning resources for our global health clients and have traveled to work directly with our target audiences in Tanzania, Ethiopia, Kenya, Sierra Leone and Ghana. 

We have built a working prototype to understand the potential and constraints of generative AI technology in our context. We have loaded only a portion of our training content into the tool to conduct our first round of testing. We have begun testing internally by asking questions and verifying the content of the answers, but we have not yet shared with an external audience. Since we are dealing with medical information, we would like to do significant testing and training on the model before sharing externally. 

We are applying to Solve to expand our reach, elevate our technology, and amplify our impact. We have created a tool that could be a game-changer for countries, especially those with remote, fragile, or conflict areas where it is difficult to reach health facility staff in-person for training and support. 

We have three main goals for applying to Solve:

1.     Strengthen our monitoring and evaluation – To demonstrate impact on performance and long-term skill retention, we need a comprehensive monitoring and evaluation plan. This piece is often not funded when we implement with our country partners. It is also the piece that could help unlock more opportunities because country governments and funders want to see impact data in order to inform their funding decisions.

2.     Learn from others – As a learning organization, we deeply value learning from others’ experiences, both what has worked well and what has not. We see being a part of the Solve cohort as an opportunity to both learn from others deploying digital tools in fragile and conflict areas, especially in an age of evolving digital privacy requirements, and share our experiences as well.

3.     Advance our business model – We have been implementing our business model since 2021 and have largely relied on the strength of our existing partnerships. We have created a model that we think responds to what the “market” of country governments and other immunization partners wants and can fund. However, it works on a project-by-project basis. We would like to reimagine our strategy for the decade ahead.

Traditional training and support for country immunization programs relies on two main methods: in-person cascade training and in-person supportive supervision visits from a manager. In the former, representatives from health facilities are taken out of their workplaces, asked to sit through days of didactic training, and then receive little to no follow-up to apply what they have learned when they return to the job. For the latter, visits from a manager are often based on convenience rather than need, if they happen at all. And in some fragile or conflict settings, it is difficult to reach health facilities if there are safety and security concerns.

With Immunization MentorChat, health workers receive adaptive, personalized support in the flow of their work. They're able to use their own natural language to ask immunization-related questions and receive trustworthy, personalized answers based on the most current, globally approved guidance. Healthcare staff can learn and be supported without traveling away from their clinics for training. In fragile or remote areas in which training or supervision are hindered by security concerns or geographic conditions, IA can bring more personalized training to places that desperately need it and have not received it in a while.

We've tried to address this question below. Apologies for not having a more robust answer here!

Our impact goals are to improve the skills and knowledge of our target learner population and to improve efficiency and accessibility of training in the country. A capable, supported and motivated immunization workforce could improve immunization coverage and health outcomes. Therefore, we ultimately hope to improve service delivery and program management. 

Measuring our progress toward these goals is challenging, since many factors impact a person’s ability to perform at their jobs; training is just one. In the past, our in-country partners have helped us implement observational assessments, to measure the impact of Immunization Academy and IA Learn. In one observational study, within 15 months of implementation, health workers who used Immunization Academy tools had greater improvement in their overall immunization knowledge and practices. Additionally, we have remotely used a pre- and post-test model to measure the impact of our training. In one efficacy study, it was found that Immunization Academy videos increase healthcare worker knowledge by 25%. 

We look forward to learning of innovative approaches we might use to measure the impact of our MentorChat solution. 

Ultimately, our goal is to reach more health workers – especially those in hard-to-reach, fragile, or conflict areas – with the tools and information they need to do their jobs well, starting with immunization and expanding to primary health care. We want to contribute to a health workforce that is resilient, can upskill and respond quickly to the next pandemic or natural disaster, and supports health workers with the resources they need to feel capable, confident, and motivated.

Immunization MentorChat is powered by ChatGPT and DocsBot AI, and is integrated with our propriety digital learning platform, Immunization Academy. We create video lessons in multiple languages and have begun using AI-supported translation and video-editing tools to reduce costs and gain efficiencies in our production process.

Full-time staff: 20

4 months

Though BCL does not currently have a formal DE&I policy, the organization is committed to creating a diverse, inclusive, and psychologically safe workplace. 

To support the freely available Immunization Academy and Immunization MentorChat, our IA Learn app is designed to be an affordable and sustainable, custom training solution. The IA Learn app is available to country immunization programs on a yearly subscription basis. We start by working with a country to determine which courses will be most relevant for their workforce based on training needs or programmatic priorities, and then we build a training curriculum using our library of video-based lessons. Once the curriculum is developed, it can be deployed through the IA Learn app to as many learners as the country aims to train within the year. This means that the cost per learner decreases as more learners are added. We also create bespoke training solutions for the global health community outside of our Immunization Academy suite.

Building on partnerships established over the years working in global immunization and workforce development, we have brought in six new funding partners with whom we have implemented fifteen projects. For example:

https://immunizationacademy.com/pdf/IA_CaseStudy_DRC.pdf