SEMA Health Feedback System

Public health care systems in East Africa are put under an enormous pressure to continue to provide good health care to an ever growing population. Approximately 88.13% of Ugandans living below 2 USD a day access health care through government facilities each year, which are at least 12 million people. Kenya faces similar challenges. There are about 4,700 health care facilities in Kenya that cater to the population of 44 million residents. The public sector serves more than half of these residents and accounts for about 51% of all health care needs. The reason public health care takes precedence over the private sector is the affordability at government owned health care facilities as prices are greatly subsidized, some services are even offered for free to residents.

At the same time, public hospitals in Uganda and Kenya are severely under-resourced and provide services that are inefficient and corrupt.  These challenges include long waiting times, lack of medicines, lack of attention or respect from staff and absent doctors. The COVID pandemic put an additional burden on the health care system: while serving more patients, protective measures and operating procedures have to be followed, which further affects the work environment of healthcare professionals. But if these professionals were incentivised to create a client-oriented environment, many challenges could be solved. 

There is also a lack of effective communication between communities and the health facilities that serve citizens. According to research conducted by the Uganda National Health Users’/Consumers’ Organization (UNHCO), there is low awareness of existing feedback mechanisms for patients (primarily suggestion boxes), many facilities don’t have a suggestion box at all and in places where they do exist many are never opened,. which deprives communities of the right to participate in the improvement of the services they receive and assume their position as stakeholders.

Equipped with the feedback data to find out which services and departments need improvements, health administrators can make a lot of changes to improve the quality of the services provided. The problem we’re trying to solve is that health facilities currently don’t use citizen feedback to improve their service quality, and are not motivated to make those improvements. 

Our feedback tools allow any citizen to give feedback on their experience at a health facility. Evidence has shown our solution has a positive impact on the behaviors of frontline workers at health facilities. The solution SEMA provides equips public health facilities in Uganda and Kenya with the data they need to change staff behaviors that improve service delivery to citizens. Some behavioral impacts our citizen feedback system improve include but are not limited to health facilities opening on time, more readily available doctors (reduction in absenteeism), less discrimination and corruption, and also guiding citizens with the right medical knowledge and according to the procedures, better sanitation - all things that are in the power of health administrators to change and can have an impact on the service quality. 

Functionally, the data we collect with our solution allows health facilities to (a) find out how their services are perceived by citizens in real-time through our online dashboard (b) demonstrate through our monthly reports and in-depth discussions how specific facilities and departments can improve their performance despite resource constraints (c) evaluate and reward staff performance based on scores and (d) monitor the effectiveness of service improvements over time. 

Our citizen-feedback system and data-to-action methodology has been developed and tested at various Ugandan government services since 2018. The evaluation of our system, conducted by the Busara Centre for Behavioral Economics and an LSE PhD researcher, has shown that SEMA’s intervention designed to improve accountability plays an important role in improving staff behaviours and service delivery, even after resource and capacity constraints are addressed. SEMA’s system works by establishing incentives for healthcare providers to perform better, while also creating an evidence base so facility managers can lobby for more resources. 

• The SEMA system consists of a combination of hardware and face-to-face data collection, which is designed in a way to generate a large quantity of real-time data, as well as action-oriented qualitative data on a daily basis.

  • We create feedback terminals with local recycled plastics at 125 USD each. Users of public services press happy or sad faces to register feedback. They have already been used in Uganda by over 100.000 citizens. This generates real-time high quantity data, especially from illiterate people, people without cell phones, and people speaking minority languages. 

  • To add more richness to the data, SEMA trains University students to do representative samples of face-to-face interviews. These interviews identify problem areas that our buttons cannot, such as waiting times, discrimination practices and actionable recommendations for service improvement. We have by now trained over 80 students in conducting interviews, 70% of which have found jobs within 6 months after their graduation from the traineeship programme with SEMA. Doing over 100 interviews per office each month costs us less than 1000 USD a year.   

• After we collect feedback, we incentivise improvements at health facilities by leveraging behavioral science methods, such as the delivery of monthly customized feedback reports, a ranking- and rewards system, and assigning champions within facilities to spearhead improvements. This is the key impact SEMA drives, behavioral change at health facilities can have a big impact on a citizen’s day to day life, for instance if doctors have better behavioral incentives to provide good service then a citizen is less likely to waste money and time visiting the same facility multiple times to find an available doctor that can help them.

• Last but not least, we continuously track the impact of service interventions at health facilities, which helps them understand what works and what doesn't. 

Existing alternatives to giving feedback at most public offices for citizens in East Africa include suggestion boxes, which the World Bank notes are “reportedly not systematically analysed or acted upon.” Secondly, government bodies regularly deploy performance/household surveys, conducted on average 3 years and costing millions of dollars. Neither alternative provides real-time or office-specific data. Tech platforms that have tried to launch in Uganda or Kenya and failed include IPaidaBribe, AskYourGov, and WeSpeak. These platforms have failed either because of 1) low usage by citizens, often by taking an overly-digital approach; or 2) inability to turn feedback into actual service improvements.

Health services touch every single citizen. Millions of Africans are pushed into extreme poverty every year because of out-of-pocket and outpatient health expenses. Poor health services have been cited as one of the major challenges that Kenyans, Ugandans and Africans as a whole face. These challenges include long waiting times, lack of medicines, lack of attention or respect from staff and absent doctors. There is also a lack of effective communication between communities and the health facilities that serve citizens. Structures for data collection, reporting, and feedback exist at every level of the health care system, however their level of functionality is marginal.

At the primary health care level, where all data are captured and generated, health workers lack capacity in terms of skills to record accurate data, conduct the analysis, and effectively use the information for decision-making. According to research conducted by the Uganda National Health Users’/Consumers’ Organization (UNHCO), there is low awareness of existing feedback mechanisms for patients (primarily suggestion boxes), many facilities don’t have a suggestion box at all and in places where they do exist many are never opened, which deprives communities of the right to participate in the improvement of the services they receive and assume their role as stakeholders.   

It is exactly these facilities that do not have access to data on their own performance, don’t have enough staff and lack morale to improve their service delivery. The healthcare sector is also faced with the problem of well trained personnel immigrating due to the low salaries and too much work. This leaves fewer personnel to attend the patients therefore increasing waiting times in outpatient departments.

The Ministry of Health in both Kenya and Uganda is however very keen on keeping staff motivated and monitoring their performance, they just don't have the resources to do so. Their interest in SEMA is high because we are the only provider who can offer them a competitive solution. At the same time, citizens visiting these facilities do not have a voice in evaluating them and feel disempowered. Bribes continue to be inherent in service delivery and disproportionately impact the poorest. SEMA has evidence that our presence reduces service-level corruption rates by 20% and has a direct effect on absenteeism, waiting times, friendliness of staff and discrimination. All of these elements affect the lives of people visiting public health clinics. It will make the service quality much better if the clinics are able to monitor their performance and encourage service improvements.

Although our feedback system primarily focuses on residents accessing  level 1 to 3 health facilities in urban areas, we are seeking more funding and resources to venture into more rural hospitals in the future.

SEMA is a team of community activists, feminists and civic leaders. The core team currently consists of 7 full-time team members and 3 part-time employees, mostly based in Uganda. We have one Operations Manager in Nairobi who is building our Kenyan office and has extensive experience working with data and government partnerships. We also currently have over 60 data collectors who are engaged with SEMA on a Traineeship contract and receive reimbursements for each shift they do to collect data (both in Uganda and Kenya). SEMA’s Board of directors consists of 4 non-executive international directors, and 2 executive international directors. SEMA also has an Advisory Board of 5 Ugandan experts. 

The SEMA team has combined experience of over 40 years and is dynamic and talented in building and sustaining an innovative solution for community development. Every team member is involved directly from the start of a journey with a public institution until the point of measuring impact. Each team member comes from a different cultural and educational background and hence is able to bring diversity to the team. 

The team has developed skills over time with the engagements of different public institutions which include, technology development, citizen & public officer engagements, research and impact tracking.

The SEMA team will deliver and scale this product and take it to the next level because the team takes joy in learning what works and what doesn’t when trying to improve public service delivery and since 2018 the team has continuously tested various data-to-action strategies that provide different incentives to public offices in order to improve service delivery, foster transparency in public offices and enhance mobility from different stakeholders.

This is a pivotal moment for SEMA to test and transition its model to serve at a bigger scale with public health facilities in Uganda and Kenya. The budget we’re looking for in order to scale up across 50+ health facilities in both countries, benefiting 1 million citizens after 3 years, amounts to € 100,000 USD. We expect to use this budget in a period of 18 months to also continue to improve our data collection/delivery system, to lower the cost of our product, expand our ‘sales’ in both the public and private sector, and with our results build on behavioral science research in partnership with the Busara Centre for Behavioral Economics.

Existing alternatives to giving feedback at most public offices for citizens in East Africa include suggestion boxes, which the World Bank notes are “reportedly not systematically analysed or acted upon.” Secondly, government bodies regularly deploy performance/household surveys, conducted on average 3 years and costing millions of dollars. Neither alternative provides real-time or office-specific data. Tech platforms that have tried to launch in Uganda or Kenya and failed include IPaidaBribe, AskYourGov, and WeSpeak. These platforms have failed either because of 1) low usage by citizens, often by taking an overly-digital approach; or 2) inability to turn feedback into actual service improvements.

Health facilities hardly ever receive feedback on their own office performance and therefore don’t feel accountable to the needs expressed by citizens. No change is expected to happen at the service delivery level if civil servants are not addressed directly and incentivised to improve. When perceived criticism on public service is handled fairly, transparently and in a timely manner, reputation and image are enhanced, staff morale improves, public confidence in government grows and most importantly, public servants know what changes and interventions are necessary to improve public services.

SEMA introduces feedback tools at local public offices, such as health facilities, municipality offices and police stations. We gather feedback through two ways. Firstly, we deploy locally-produced IoT feedback devices, which by one press of a button (a happy or sad smiley face) allows any citizen, regardless of their literacy level or mobile phone usage, to give feedback. Secondly, we conduct on-site in-person interviews through a student-run network of data collectors. Through these interviews we gather deep and rich feedback data in local languages on why citizens are happy or unhappy with the service. Every month, we present the feedback to local civil servants in one-pager score reports with rankings. Our behavioral science approach and impact are what makes our solution unique and powerful.

Through SEMA, citizens have an active voice in evaluating their local public institutions, such as critical justice and health services, and hold them accountable to provide fair and nondiscriminatory services. The result based on this feedback loop are behavioral changes at the staff level that improve service delivery at health and other critical service facilities citizens rely on every day. We expect in the long-run our solution will highlight the importance of behavioral science mechanisms to improve service delivery and create more incentivized, accountable and transparent critical public and private services.

1. Ensure transparent and accountable institutions at all levels of government (SDG 16.6). 

2. Improve the confidence citizens have in sharing their opinion about health service delivery, by showing them that feedback matters. The next time they visit a local hospital, they will notice the service has improved. Our expected outcomes therefore are an increased number of citizens sharing their feedback at hospitals, and improved health service delivery at more public health facilities in East Africa. 

Impact goals 2020-2022

Note - in reference to citizen engagement: We have already surpassed our goal of reaching 100,000 citizens significantly in 2022. Our revised goal for 2023 is to reach over 280,000 citizens across Uganda and Kenya.

Long-term Goals 2022-2023

We aim to be in 300 offices by 2025.

Approaches 

We are taking four main approaches to support our funding and growth objectives for the next 2 to 3 years (2022-2025):

1. Government Partnerships (Approach 1): Continue strengthening and developing our public partnerships in Uganda and Kenya. We aim to have gained at least six new public partnerships with different government entities in Uganda and Kenya (3 Kenya, 3 Uganda) by Q4 2023.
2. Government Procurement & Fundraising (Approach 2): Diversify sources of funding to reduce revenues coming from grants/donations by 15% by December 2022. 

• Activities: 
  • A. New public partners as discussed under outcome 1 will add to the diversification of SEMA’s revenue sources as they start including SEMA in their annual budgets and paying out procurement contracts. No funds will be used to directly subsidize a sales programme for devices, and if any SEMA devices are sold to for-profit companies, all profits will be used to support the charitable activities of SEMA. 
  • B. Ongoing fundraising with international partners and philanthropists to secure SEMA’s overhead funding will continue globally by SEMA’s management. 

  3. Thorough RCT Study to evaluate and improve SEMAs impact (Approach 3)

• Activities: 
  • This study will aim to answer the following research questions:
    • Research Question 1: What is the causal impact of SEMA’s feedback system on client satisfaction and improved services at health facilities in Uganda? 
    • Research Question 2: How long is any effect on client satisfaction and improved services present for?
  • Secure investment to fund RCT - We are currently in semi-final stages of a potential investment to fund this RCT. (Investment fund is now conducting due diligence before final decision is made)

4. Independent and Private institution Procurement (Approach 4): Conduct private sector research and lead generation to determine opportunity, need and modelling of SEMA citizen feedback services for private and independent  institutions, our primary focus in this sector is on private health care institutions. We are already engaged with the likes of the Huduma Kenya, Marie Stopes Kenya and International, Jacaranda Health and Elephant Healthcare to name a few institutions interested in the feedback system SEMA provides. 

Theory of Change:

The SEMA feedback mechanism supports achieving the long term objective of improving the quality of services at public institutions, in particular health facilities. Causal chain:

Ultimately we find that the key to creating impact in the public sector is the relationship building and hyperlocal reporting that happens at the local office level. It is extremely rare in East Africa that feedback is given directly to the local office. Most public sector assessments are done at a national level, and delivered to HQs. What sets the SEMA feedback system apart is that it is entirely built on a hyper-local approach.

By gathering qualitative and quantitative feedback data from citizens, and delivering it in easy-to-understand reports and by creating behavioral incentives to local civil servants at health facilities, we are able to create service-level improvements executed by the local offices, leading to improved service delivery experienced by more citizens through reduced wait times, bribes, less discrimination and more resolved problems.

Our solution is a hybrid of an IOT and SaaS application. The SEMA feedback device is an IOT device with a five point scale where clients in public offices can rate the service provided at these institutions. The data is stored on the device and then uploaded to the cloud servers(databases). The other part of the feedback system is the dashboard which uses the Saas architecture. The dashboard is used internally by the team to maintain the devices and externally by the public institutions to track client satisfaction in real time. The SEMA dashboard uses the data that was uploaded to the cloud servers from the feedback device to break down data into different insights that can enable public officers to make better decisions on improved services.

The code for all our applications is internally maintained by the technology team and is not open source.

The SEMA feedback system consists of a combination of hardware and face-to-face data collection, which is designed in a way to generate a large quantity of real-time data, as well as action-oriented qualitative data on a daily basis.

1. We create feedback terminals with local recycled plastics at 100 USD each. Users of public services press happy or sad faces to register feedback. They have already been used in Uganda by over 200,000 citizens. This generates real-time high quantity data, especially from illiterate people, people without cell phones, and people speaking minority languages.

2. To add more richness to the data, SEMA trains University students to do representative samples of face-to-face interviews. These interviews identify problem areas that our buttons cannot, such as waiting times, discrimination practices and actionable recommendations for service improvement. We have by now trained over 50 students in conducting interviews, 80% of which have found jobs after their traineeship programme with SEMA. Our trainees are paid a monthly stipend, doing over 100 interviews per office each month costs us less than 1000 USD a year.  

3. After we collect feedback, we incentivize improvements at public health offices by leveraging behavioral science methods, such as the delivery of monthly customized feedback reports, a ranking- and rewards system, and assigning champions within facilities to spearhead improvements.

4. Last but not least, we continuously track the impact of service interventions at public health facilities, which helps them understand what works and what doesn't.

SEMA has an international board of directors, an advisory board of senior Ugandan government and INGO experts, and an executive team consisting of 12 amazing minds with over 50 years of combined experience. The social enterprise was founded by three social entrepreneurs, including a Dutch data- and impact measurement expert, an American civic tech entrepreneur and a Ugandan lawyer.

SEMA’s data collection is supported by a network of 47 student data collectors (35 in Uganda and 12 in Kenya) and various consultants in software engineering and graphic design.

Overall, SEMA’s operational team is 80% women and 20% men, and a majority of the team is Ugandan.

Fundraising Goals and Approaches: 

Goals

We aim to focus on three main activities that will become sustainable.

Firstly, by investing in a business developer in Kenya who actively develops (paid) government partnerships and fundraises for grants, we give ourselves some time to get in a bigger funding opportunity to support our work in Kenya.

Secondly, by investing in a standard curriculum for our data collectors, we will have a handbook, set of training in powerpoint, systems and methods to mentor and monitor our data collectors. We expect the deliverables of this curriculum to be replicable and therefore relevant beyond the term of this project.

Third, by running a comprehensive RCT to evaluate our impact and improve on our solution and finally by piloting mobile surveying techniques, we become more resilient as an organisation to continue to deliver results and impact, even in times of insecurity or lockdowns. This will make us sustainable in the long run. 

Approaches

Four main approaches will support these funding and growth objectives for the next 2 to 3 years (2022-2025):

1. Government Partnerships (Approach 1): Continue strengthening and developing our public partnerships in Uganda and Kenya. We aim to have gained at least six new public partnerships with different government entities in Uganda and Kenya (3 Kenya, 3 Uganda) by Q4 2023.
2. Government Procurement & Fundraising (Approach 2): Diversify sources of funding to reduce revenues coming from grants/donations by 15% by December 2022. 

• Approach 2 Activities: 
  • A. New public partners as discussed under outcome 1 will add to the diversification of SEMA’s revenue sources as they start including SEMA in their annual budgets and paying out procurement contracts. No funds will be used to directly subsidize a sales programme for devices, and if any SEMA devices are sold to for-profit companies, all profits will be used to support the charitable activities of SEMA. 
  • B. Ongoing fundraising with international partners and philanthropists to secure SEMA’s overhead funding will continue globally by SEMA’s management. 

  3. Thorough RCT Study to evaluate and improve SEMAs impact (Approach 3)

• Approach 3 Activities: 
  • This study will aim to answer the following research questions:
    • Research Question 1: What is the causal impact of SEMA’s feedback system on client satisfaction and improved services at health facilities in Uganda? 
    • Research Question 2: How long is any effect on client satisfaction and improved services present for?
  • Secure investment to fund RCT - We are currently in semi-final stages of a potential investment to fund this RCT. (Investment fund is now conducting due diligence before final decision is made)

4. Independent and Private institution Procurement (Approach 4): Conduct private sector research and lead generation to determine opportunity, need and modelling of SEMA citizen feedback services for private and independent  institutions i.e offering procurement of SEMA feedback devices, citizen feedback/data consulting etc.

Our primary client focus in the private sector are health care institutions. We are already engaged with the likes of Marie Stopes Kenya and International, Jacaranda Health and Elephant Healthcare to name a few institutions interested in the feedback system SEMA provides. 

Progress to date - Grants raised to date (link here)

• Since 2018, SEMA has run a successful pilot of its citizen feedback tools and evaluated its data-strategies in Uganda with the help of the Busara Center for Behavioural Economics. As a result, 100,000 citizens gave feedback in a period of three years at over 26 different government facilities, leading to 67% of public offices with improved service quality and 80% with reduced waiting times after 3 months. SEMA has been procured by three government entities to date in Uganda, totaling nearly 100.000 USD in government contracts. We were the youngest organisation ever to win the International Anti-Corruption Excellence Award in 2019 in the category of innovation. 

• In 2020, when the COVID pandemic just started, the Kampala Capital City Authority asked SEMA to start measuring the performance of 6 key health facilities in Kampala. This successful pilot led them to procure SEMA again in 2021. The Ugandan Ministry of Health is keen to develop an MoU with SEMA to expand to more offices.

• We ended 2021 with a view to international expansion: we signed an MoU with the Nairobi City Council with a prospect of starting a pilot in Q1 2023. 

• In partnership with Busara, we have run two evaluations in 2019-2020, thereby a.o. testing the effectiveness of our ‘champions’ intervention model. A PhD economics researcher from LSE is in the final stages of publishing a paper on the effectiveness of SEMA as an accountability mechanism, controlled for resource constraints at pubic facilities. In addition, we are preparing for a RCT study of SEMA’s system in Q1 of 2023.