Finalist
2021 Digital Workforce Challenge

KOSA AI - future proofing AI

Team Leader
Layla Li
Solution Overview
Solution Name:
KOSA AI - future proofing AI
One-line solution summary:
KOSA AI builds multi-stakeholder AI governance software targeted at detecting and mitigating AI bias throughout the ML lifecycle.
Pitch your solution.

Artificial intelligence (AI) constitutes an essential part of today’s economy, but is riddled with biases that disadvantage historically underserved groups unfairly. In an age where workplace practices are becoming more and more digital, the proliferation of workforce management systems, such as applicant tracking systems (ATS) -such as resume scanning increases the risk of AI biases in hiring, retention, and promotion practices across organisations. 

At KOSA AI, we have developed a proprietary automated responsible AI system (ARAIS) to help businesses identify and minimise biases inherent in their AI-powered decision-making models. Our solution is sector-agnostic and has multiple use cases including ATS. By identifying and mitigating biases in AI and ML processes across industries, our solution will reduce the hidden inequalities that today affect millions of people and will enable equitable access to jobs, irrespective of race, gender, age or other social identifier. 

Film your elevator pitch.
What specific problem are you solving?

In today’s economy, AI powers thousands of decision-making tools, yet its biases against historically underserved groups such as women and other minority groups is often misunderstood and underestimated. These can be found in applications within human resources across multiple industries. 

KOSA AI is working on detecting and mitigating the bias within ATS use cases, where AI bias is denying 27 million people from finding new jobs as the biased hiring algorithm focuses on credentials rather than capabilities. 75% of employers rely on AI tools to scan resumes and as more organisations adapt to the perpetual growth of technological change, specifically, the harder it is for the workforce to keep up to date with their skills. Across hiring algorithms, this bias is particularly pervasive. For instance, Amazon was forced to renounce the development of a proprietary hiring system because it showed that it repeatedly discriminated against women. It specifically penalised resumes that included the word “women’s” and downgraded graduates of two all-women’s colleges in the US. This was a result of the way Amazon’s computer models were trained to vet applicants by observing patterns in resumes submitted to the company over a 10-year period, most of which came from white males.

What is your solution?

KOSA AI helps organisations improve hiring, retention, and promotion practices by mitigating biases present in AI decision-making tools. Our proprietary AI governance software solution supports organisations to recruit and retain a more diverse and inclusive workforce, which has been shown to provide countless benefits to businesses, from identifying missed revenue opportunities through increased customer trust and better product development to reducing costly litigation risks and more. 

Our automated solution seamlessly integrates into an organisation's existing AI infrastructure. Our product comprises of 4 steps that provide support across the whole ML lifecycle: (1) it assesses and mitigates the biases in current ML processes; (2) it audits the AI model to assess the human impact and to automate compliance checks; (3) it explains the model’s behaviours; (4) it builds an additional monitor module to track drifts or malfunctions and allow developers to fix vulnerabilities. Building responsible AI is a team effort; therefore we have developed tools for all stakeholders, from the executive team to the developers. Our software outputs both an evaluation for the technical development team to understand biases within their systems and a quantifiable financial assessment for non-technical stakeholders to grasp the missed opportunities for the business. 

Who does your solution serve, and in what ways will the solution impact their lives?

KOSA AI’s mission is to make technology accessible and inclusive of all people. We work to support SDG 10 to reduce inequality. By 2030 we strive to empower and promote the social, economic and political inclusion of all, irrespective of age, sex, disability, race, ethnicity, origin, religion and economic or other status. As we target companies using AI at scale irrespective of industry, we aim to serve the global digital workforce and traditionally underserved groups who have been denied equitable access to job opportunities in particular.

Due to the nature of our solution, we work with enterprises directly and not our target beneficiaries. However, we do everything we can to ensure that our software has the impact we aim to achieve. For instance, using the use case of ATS, we conducted a proof of concept by automatically neutralising subjective bias in text within resumes. We used a Bias Statement Detector (BSD) to detect and quantify the degree of bias within the sentences and could verify that the model is able to consistently predict the average bias achieving greater than 97% accuracy. This confirmed the inherent bias that lies within text and the impact keywords and language has on the degree of bias within the resume. 

Furthermore, we work with a number of research institutions such as the University of Massachusetts Amherst (UMASS), Institute of Electrical and Electronics Engineers (IEEE), Porsche and the Linux Foundation  to increase knowledge and awareness of AI bias mitigation and ethical bias; we want to make sure that AI-driven inequalities are better understood to develop solutions that respond to the needs of those impacted. 

In efforts to speed up the reduction of AI inequality, be it with the companies KOSA AI works with directly or with our target beneficiaries, KOSA AI is building an AI Academy. Here we provide course material and conduct workshops that are used to educate and create awareness on specific research topics such as ethical AI, AI governance, AI fairness definition, bias within facial recognition software, other computer vision applications and more. Through this platform, we endeavour for our target population to have access to improved quality data, help shape a supportive digital policy environment that enables data sharing whilst protecting privacy and security, and define use cases where AI has the largest potential for impact. One of the key locations for the AI Academy’s deployment is in Africa. 

KOSA AI has a prominent presence in Africa and here we leverage our distributed model to improve our software and increase our impact. Our network on this continent favours (1) access to diverse training data sets which we can share with companies outside of the continent who require more representative datasets; and (2) build a data bank with more diverse data sets; and (3) increase the diversity of our team by mobilising African talent. In Africa, we see an opportunity to directly grow KOSA AI’s vision and create tangible impact, especially for rural communities and historically underserved groups. Specifically in Kenya, where the emerging technology sector is rapidly growing into a consequential share of national GDP, the importance of AI governance within the tech ecosystem is crucial. KOSA AI’s AI academy, research partnerships with organisations such as the Technical University of Kenya (TUK), and other business efforts will reduce inequalities in the digital workforce by empowering minority groups and the underserved population through education, awareness and training on responsible AI and AI bias.

Which dimension of the Digital Workforce Challenge does your solution most closely address?
  • Reduce inequalities in the digital workforce for historically underserved groups through improved hiring and retention practices, skills assessments, training, and employer education and engagement
Explain how the problem you are addressing, the solution you have designed, and the population you are serving align with the Challenge.

The Challenge seeks solutions that will help underserved communities be better prepared for success in the digital workforce. It requests solutions that are technology- or innovation-based and use data science for positive change to support these communities. Our solution is a technology-based innovation that enables companies to identify and correct biases in their AI and ML that prevent minority groups such as women and historically underserved groups from accessing jobs they would otherwise be qualified for. Through our technology, businesses across industries can improve their hiring practices to include and support the underserved. 

In what city, town, or region is your solution team headquartered?
Amsterdam, Netherlands
Is your solution already being implemented in one or more of the following ServiceNow locations (Australia/New Zealand, Canada, France, Germany, India, Ireland, Israel, Japan, the Netherlands, Singapore, the United Kingdom, United States), or are you planning to expand your solution to one or more of these countries?
  • My solution is already being implemented in one or more of these ServiceNow locations
What is your solution’s stage of development?
  • Pilot: An organization deploying a tested product, service, or business model in at least one community.
Explain why you selected this stage of development for your solution.

KOSA AI launched a minimum viable product (MVP) earlier this year and has secured Letters of Intent from 5 customers across healthcare, technology, services, government, and education. We expect to convert at least 2 of our 5 piloting organisations into paying customers and plan to fully launch our product by early 2022. In addition, we are currently working on a community launch with Ai4Gov to deploy govtech solutions designed to improve the business environment in the Philippines, with the common objective to improve, develop, and promote, directly or indirectly, ethical AI practices. Finally, we have secured partnerships with Porsche, Linux Foundation, IEEE and TUK to perform extensive research into topics such as ethical AI, AI governance, AI fairness definition and more. The results from this research will contribute to KOSA AI’s architecture, long term development and subsequent success. 

Who is the Team Lead for your solution?
Layla Li
Do you qualify for and would you like to be considered for the ServiceNow US Racial Equity Prize? If you select Yes, explain how you are qualified for the prize in the additional question that appears.
  • Yes, I wish to apply for this prize
Explain how you are qualified for this prize. How will your team use the ServiceNow Digital Equity Prize to advance your solution?

The ServiceNow Digital Equity Prize seeks solutions that address communities of colour in the US, with a special focus on an equitable digital workforce. By identifying and mitigating biases in AI and ML processes across industries, our solution will reduce the hidden inequalities that today affect millions of people and will enable equitable access to services and jobs they want, irrespective of race.

Though we do not directly work with our target beneficiaries, we enable businesses to improve the quality of data that is fed into their decision-making tools to make them fairer while increasing their revenue stream; thanks to us, companies can be more profitable by being more equitable. We expect to mainstream AI data fairness across organisations and sectors, catalysing broader positive impact for all those impacted by our customers products and services. This includes people of all races and backgrounds. 

As seen above, UnitedHealth’s Optum algorithm, serving 200 million people, showed drastic racial bias, denying care to 50% qualifying black patients. At KOSA AI, we seek to reduce these inequalities by equipping businesses with our AI bias detection and mitigation technology. With support from this prize, we can accelerate the deployment of our technology across services sectors to improve access to financial services, insurance and health care services and strengthen economic opportunities for thousands of underserved communities across the US and globally.

More About Your Solution
About Your Team
Your Business Model & Partnerships
Partnership & Prize Funding Opportunities
Solution Team:
Layla Li
Layla Li
Co-founder & CEO
Sonali Sanghrajka
Sonali Sanghrajka
Co-Founder & CCO