"Know-it-Or-Not"

Vaccine hesitancy is a complex global phenomenon, and programs that treat hesitancy as a problem of information deficiency do not work – political ideology, legacies of structural racism, and other psycho-social factors all affect how people consume and understand health information.

Our solution - “Know-It-Or-Not” (and predecessor "It’s Contagious") are gamified digital interventions built to engage users on misconceptions related to COVID-19 and vaccines. Adaptive learning profiles tailor content to the unique vulnerabilities to misinformation of each user, and are developed in partnership with behavioral scientists to promote the cognitive processes that mitigate biased reasoning. Playful design and gamified incentives further promote positive affect and intrinsic motivation. 

Our work with 190,000+ people demonstrated that Know-It-Or-Not significantly improves health literacy and intent to receive a COVID-19 vaccine, and we know this tool can be deployed at the scale and speed needed to make a meaningful difference in the fight against COVID-19.

Widespread vaccine hesitancy is a complex global phenomenon that threatens to undermine efforts to gain control over the COVID-19 pandemic. For example: It is estimated that up to a quarter of Americans do not plan to be vaccinated - and in some countries, population-level vaccine acceptance is as low as 15%. A key contributor to this issue is rampant misinformation campaigns occurring online, against which the susceptibility of an individual is affected by a range of psychological, socio-economic, and environmental factors.

Importantly, when these factors are not addressed, misinformation interventions typically fail to have an effect and even backfire. Treatment of vaccine hesitancy as a problem of information deficiency alone overlooks key influential factors including user values, affective and motivational reactions, along with contextual barriers stemming from historical mistrust of medical institutions in marginalized communities. 

Globally this problem is pervasive amongst healthcare providers in formerly colonized settings; for instance, a recent study in Nigeria found vaccine acceptance at 28% amongst healthcare workers. Overcoming vaccine hesitancy is therefore even more critical as the world focuses on ending both the COVID-19 pandemic and decolonizing public health.

Our goal is to help improve COVID-19 vaccine acceptance rates globally, by strengthening health literacy and resilience to misinformation.

"Know-It-Or-Not" (and predecessor "It’s Contagious") are gamifed digital interventions that promote health literacy by targeting the diverse array of psychological biases and strongly held beliefs that make a person vulnerable to misinformation around COVID-19 and vaccines.

Our platforms are designed using principles of cognitive theory to mitigate negative feelings and biased reasoning that often get in the way of revising misconceptions and accepting corrective information. The content is tailored to users' unique “vaccine hesitancy profiles”, powered by machine learning and human centered design. Targeting individuals with customized information promotes enduring learning effects, and provides a flexible approach for deployment across contexts where reasons for hesitancy differ and user customization is key. 

Platform content is updated continuously based both on new information about the disease and vaccine, and on trends in emerging misinformation narratives. Updates and new content are tested with experimental rigor before implemented at scale, in collaboration with leaders and organizations representing vulnerable communities.

We anticipate our tool can be adapted in the near future to address a broader range of health information topics beyond COVID-19.

Our target audience are individuals that are vulnerable to misinformation about COVID-19 and/or are vaccine-hesitant. Unfortunately, these populations tend to have the greatest incidence rates of COVID-19, and across the USA and Canada are disproportionately represented in communities of color and those that are underserved. Globally, our target audience extends beyond the general public, with vaccine hesitancy prevalent amongst healthcare providers and complicated in settings where governments themselves may be perpetuating false narratives.

Our baseline work included developing a vulnerability assessment to identify themes and factors affecting an individual’s vulnerability to misinformation, consulting groups on content, and then finally piloting the app with these communities to assess whether we were achieving robust learning effects across diverse groups. We plan to take a similar approach with new communities/regions in this next phase, and as we continue to scale, deployment of our vulnerability assessments will help us strategically define our target users for every new context we enter.

Altogether, this solution aims to improve vaccine acceptance rates to collectively move past this pandemic, and is rooted in equity and prioritization of vulnerable populations everywhere. 

Our program intends to fight the spread of mis and disinformation around COVID-19 and vaccines with the ultimate goal of protecting communities. Extensive research on combating vaccine hesitancy makes it clear that treating it as a problem of information deficiency does not work. Political ideology, legacies of structural racism, and other psycho-social factors affect individuals’ appraisal of health information. Instead, our tools strongly align the objective of inspiring individuals to protect themselves and our communities, by using playful and human centered design elements that tap into the cognitive processes that make people resilient to mis- and disinformation. 

By the end of December 2020, over 190,000 players in North America used It’s Contagious, and evaluated over 2.11M misinformation claims. The platform has been tested vigorously using RCT designs in Canada and the US and achieved success in improving knowledge scores in participants across the ideological spectrum, and in reducing anger/skepticism when presented with corrective information in comparison to a non-gamified version. Our product is robust, adaptable, and scalable and having gone through a vigorous development and design phase, we are now poised to start scaling globally.

We are currently exploring options to do so with partners in the US, Brazil, Myanmar, Malawi, South Africa and the Philippines: all contexts where there is high vaccine hesitancy, high social media usage, pre-existing polarization amongst the population, and historical mistrust of medical institutions and their own governments - a predictor for pervasive spread of misinformation.

Our team conducted a landscape analysis examining the different digital solutions currently deployed to tackle mis information in the context of COVID-19. Currently available solutions fall under three categories: chat-bots, general information applications, and apps that create repositories synthesizing evidence from different data streams. A common theme across these solutions is an implicit assumption that vaccine hesitancy reflects a deficit of information that can be resolved by simply providing correct information. These approaches have been found to be ineffective and even backfire in some contexts and have been criticized for overlooking influential psychological factors of message recipients, including their values and affective and motivational reactions. Our solution is unique because our content is designed to promote positive affect and deploys analytics to customize content pathways to the unique “vaccine hesitancy profiles” of each user. Not only is our platform user-driven, but our gamified approach is a novel feature that fuels intrinsic motivation by personalizing the experience with visible progress bars, challenges, and opportunities to recover status / correct answers interactively. Moreover, the highly customizable nature of the game allows for it to be deployed effectively anywhere without requiring users to provide any sensitive information.

Between August and December 2020, we engaged over 190,000 players who collectively evaluated 2.11M misinformation claims.

Over the next year we plan to scale up significantly, to reach over 5,000,000 people globally, by expanding our efforts in Canada and the US but also introducing adaptations of our program in a number of new contexts in the Global South, including: Brazil, South Africa, Tanzania, Myanmar, and Philippines. As we grow, we intend to further innovate on our platform to address vaccine hesitancy elsewhere as needed and across other disease areas and envision reaching over 25M people globally.

Our target audience are individuals that are vulnerable to misinformation about COVID-19 and/or are vaccine-hesitant. Unfortunately, these populations tend to have the greatest incidence rates of COVID-19, and are disproportionately represented in communities of color and underserved communities. Globally, our target audience extends beyond the general public, with vaccine hesitancy prevalent amongst healthcare providers and complicated in settings where governments themselves may be perpetuating false narratives.

Our baseline work included developing a vulnerability assessment to identify themes and factors affecting an individual’s vulnerability to misinformation, consulting groups on different content design elements, and then piloting the app with these communities to assess whether we were achieving robust learning effects across diverse groups. We plan to take a similar approach with new communities/regions in this next phase, and as we continue to scale, deployment of our vulnerability assessments will help us strategically define our target users for every new context we enter. 

The indicators mentioned in the previous question are all indicators that we have already been tracking through the piloting of our platforms. These will continue to be tracked in-game and will be assessed periodically to evaluate program performance.

Moreover, we plan to continue testing program optimizations and new design elements, all of which will be evaluated through best practice methods of sampling and statistical analysis to evaluate every game component and an explore-exploit framework of providing a mostly optimized experience to most users, while also injecting small changes and content updates to test new features/content. 

Key members from the Digital Public Square team:

• Farhaan Ladhani - CEO and Co-Founder
• Ben Windler - Senior Data Scientist
• Sean Willett - Product and Strategy
• Alana Changoor - Global Health Consultant

We can provide the full list of team members and affiliations upon request.

Key Partners & Advisors:

• Dr. Greg Trevors, Department of Educational Studies, USC
• Dr. Eric Brenner, M.D., medical epidemiologist and public health physician
• The Alberta machine intelligence institute (Amii)

Digital Public Square knows communities thrive when people have access to shared knowledge and when they can engage in healthy deliberation and informed decision-making.

Over the past decade, we have created an array of digital platforms that have connected tens of millions of people around the world. These platforms have been leveraged to fight COVID-19 misinformation in North America, increase religious and ethnic tolerance in Asia, and strengthen accountability in the public sector. In parallel to creating innovative engagement platforms, we have developed rigorous research methodologies that allow us to test the impact of these tools on emotional sentiment, behaviour change, and policy preferences.

During the content design phase for this initiative, we consulted with an array of people from vulnerable communities, developed vulnerability assessments to identify themes and factors affecting an individual’s vulnerability to misinformation, and piloted our gamified tool  in these communities to assess impact on learning outcomes.

As we position ourselves to scale globally, we are also working with stakeholders in the global health space, particularly on strategies to optimize distribution schemes. For instance, we are actively pursuing partnerships with local grassroots organizations (such as community health worker programs and faith-based health initiatives) to ensure our product reaches end-users in last-mile communities.

The Digital Public Square team is committed to being the model of healthy communities we want to see in the world. That means we deeply value - and actively seek - diversity, equity and inclusion for ourselves and in our practice. Our strength comes from being composed of individuals from a wide range of backgrounds and genders, with varied ethnicities, lived and professional experiences. At the time of this submission our Board and Leadership Team reflect this diversity. We also acknowledge that truly inclusive practice in hiring and work equity is never finished - it’s an ongoing process we actively address to ensure our own Digital Public Square community is well cared for and served.

Being diverse is what grounds our work and makes us stand out. Our most successful programs are built directly in collaboration with the marginalized, vulnerable, and targeted communities we aim to support.

We believe that our program offers a unique and impactful approach to improving vaccine hesitancy, that sits at the nexus of public health, cognitive science and machine learning. We believe it can deploy at the scale and speed needed to make a meaningful difference in the fight against COVID-19, and support through SOLVE could catalyze a new phase of global scale and growth in two key ways:

For one – we are keen to build partnerships where we can leverage trusted channels to reach our target end users to increase community buy-in. Specifically, we are looking for partners who have connections with communities at the grassroots level in our target countries, and whose networks can be leveraged to deploy our program. This includes community health programs and other digital health initiatives working with community leaders, which have the potential of accelerating the effectiveness of our app with target communities.

Secondly, from a technical perspective, while we expect our ability to leverage adaptive models to be a challenge, we are mitigating to some extent with the collection of context-specific training data. Resources from this program will support the solution of expected technical hurdles related to operationalizing these models and adapting them to diverse contexts in the Global South and across the US.

While our platform is uniquely well suited to scale in diverse contexts, reaching target users through strategic partnerships at the grassroots level will be critical.  This includes community health programs and other digital health initiatives working with community leaders, which have the potential of accelerating the effectiveness of our app with target communities. 

We are looking for partners to:

1. Help us optimize the adoption of this program in localized contexts. Organizations that are countering misinformation at the grassroots level, such as community health worker groups, faith-based organizations, and related health campaigns will be critical to getting our intervention to end users in the most vulnerable communities. Accordingly, we have interest in pursuing partnerships with organizations that have deep country-level networks including CHAI, PATH and the Gates Foundation.

2. Participate in a cross-exchange of learnings and expand our digital networks to communities of interest. This includes organizations that have similar values and strengths in the digital analytics space, including Facebook, Google, the Patrick J McGovern Foundation and Zenysis. 

Across the United States, vaccine supply has also begun to outpace demand, and therefore action to increase vaccination rates amongst those that remain vulnerable is more critical now than ever. The discord of conspiracies and data streams that are exploiting pre-existing distrust to fuel misinformation means that a one-size-fits-all solution simply will not work. Our solution addresses the psycho-social factors affecting an individual’s unique “vaccine hesitancy profile” using a behavioral science approach, and we leverage machine learning to make our platform deployable at scale and with speed.

An award from the Robert Wood Johnson Foundation would enable us to advance this solution by providing us the resources to continue development and expansion of our content libraries to maintain pace with the evolving information landscape, resource our outreach through advertising channels and build strategic partnerships to target underserved communities across the US. 

The world will not be rid of COVID-19 anywhere unless sufficient numbers of people choose to be vaccinated everywhere, and we believe our approach can make an impact by utilizing digital tools that integrate public health research, cognitive theoretical frameworks for understanding vaccine hesitancy, and machine learning.

Our program is very well aligned with the mission of the Patrick J McGovern Foundation, as at our core, our solution leverages machine learning to combat misinformation around COVID-19 vaccines in such a way that ensures our intervention is tailored to the unique vulnerabilities to misinformation of our users while being deployable at the speed and scale necessary to make a meaningful difference.

The AI for Humanity Award would enable us to advance this solution by providing us the resources to continue development and expansion of our content libraries to maintain pace with the evolving information landscape, resource our outreach through advertising channels and build strategic partnerships to target underserved communities across the Global South.