Codewars

In this rapidly-evolving digital economy, proficiency with software is becoming an ever more integral part of the modern worker’s skillset. In order to meet demand and ensure a diverse workforce, affordable and relevant software-related training must be provided.

Codewars makes it possible for anyone with an internet connection to participate in a learning community of over 2-million software developers. Members improve their standing in the free community by solving real-world relevant coding challenges (known as kata), authoring their own kata, and offering feedback and support to other members. Members may also complete curated learning pathways that represent specific careers in tech.

The Codewars community, ranging from novice developers to CTOs who are themselves potential employers, also benefit from access to a 3rd-party network of innovative learning and credentialing programs. Community members receive access to a dynamic, competency-based report that serves both as a learning tool and skills summary for employers.

Businesses are experiencing a digital talent gap that is not only hampering their digital transformation agendas, but causing them to lose competitive advantage because of it. A large part of this gap comes from the fact that there is untapped potential within underserved communities. In addition, diversifying teams does more than solve a shortage of digital workers, it also makes good business sense. A recent study by McKinsey discovered companies in the top quartile for gender diversity are 21% more likely to have financial returns above their respective national industry medians. And when they looked at the benefits of ethnic diversity, it jumps to 33%.

A key goal for Codewars is to work specifically with underserved communities to give them the opportunity to be successful in finding and securing software developer jobs. We work directly with The Girl Code, Detroit Public Schools, young women, minorities, etc to provide a fun and engaging unbiased mentorship environment for aspiring software developers of all backgrounds.

Codewars seeks to create a forum for mentorship and credentialing, which provide the essential confidence needed for underserved communities to realize their full potential within a digital economy.

Codewars is an online platform that allows learners to solve coding challenges and to participate in a developer community.

Coding challenges (called kata) are solved in the browser, in an environment that approximates actual development environments. Katas are available in 50+ programming languages and aim to train skills applicable in the real-world. All katas are auto-graded using unit-testing frameworks specific to the programming language the learner is coding in. This method of testing code is the de-facto standard in software.

Besides solving katas, Codewars members can participate in the community by creating new katas, translating katas into other programming languages, and offering kata-related commentary. Learners also benefit from the discussions that take place on Codewars. Support and insights that improve learning are routine.

Members may also traverse career-specific pathways that consist of a series of katas. Completion of a pathway signals proficiency in that given career track. Where appropriate, links to 3rd party education programs (MOOCS, Certification Programs) are shown.

Each member receives a profile that summarizes community participation. All activity is displayed, along with the competencies evidenced through kata completion. This profile is used as a learning tool and online resume.

Codewars seeks to increase learning access and improve employment outcomes for traditionally underrepresented groups in tech: women, those of low-income, minorities, and other underserved communities. The platform is free to use. All that’s needed to sign-up is an email, and no demographic information is required.

An emphasis is placed on skill, as ranking in the community is primarily determined through performance on katas. Katas are evaluated by the Codewars team and the Codewars community based on their ability to test and train creative problem solving skills, subject-matter knowledge, and real-world coding scenarios. Katas are also moderated to reduce bias.

The Codewars team engages with the Codewars community and select 3rd-party organizations and educational institutions such as the Detroit Public Schools system, The Girl Code, and Code2040 in order to understand how to better serve target populations. This feedback has resulted in new kata creation, existing kata improvements, improvements to the core testing technology, and changes to the Codewars platform roadmap: dynamic learner profiles and future employee and job-matching functionality being prime examples.

Codewars seeks to close the loop on the skilling and job-search process. Currently, learners on Codewars can:

• Improve their skills within an in-demand area: software development
• Receive feedback and support from a large and diverse community of like-minded developers
• Build a profile centered around a competency-model useful for learning and employment
• Progress through career pathways that approximate activities and skills required on the job

In the future, Codewars members will receive priority access to a curated list of available jobs, and can opt-in to targeted matching based on their experience and evidenced competencies.

We have twelve total staff: Two full time contractors and ten full time employees.

The Codewars team periodically reviews adherence to company conduct and hiring guidelines, which are centered around creating a diverse and equitable organization.

Our team utilizes its own technology to objectively assess potential hires. Many of our hires are made through Codewars itself. Throughout most of the hiring process, we don’t see anything about a potential hire except for their Codewars profile, which consists of their kata completed and participation in the community.

A large-part of the hiring decision is based on performance on an assessment that is mapped to job-related competencies. We use blind review (we don’t see a person’s name or email) to review the code submission.

The platform is based around small coding tasks which allows developers to attempt to solve them within the browser. These tasks require developers to have to complete the task by writing actual code. What is unique about Codewars is how the code is tested. It uses code to test code, the same way it is done when software is developed in the real world. This opens up endless possibilities of what can be tested. In addition, every coding task is created by the community, and goes through a rigorous quality assurance process which is also handled completely by the community.

While these coding tasks are the key activities performed on the site, the next stage of our Codewars journey is to build pathways which allow both aspiring and professional developers to focus on specific paths that best fit their career goals. This includes things like AI & machine learning paths, mobile development, game development and more. These paths will be anchored to our in-house developed competency framework, which has been custom developed through our sister product, Qualified.io. We have worked with hundreds of companies to help them hire and this process has allowed us to understand the exact competencies that companies are looking for. We are able to leverage this information to guide our pathways, and allow developers to discover and establish themselves in competencies that matter most to their success.

The heart of Codewars is the ability to safely execute code created by its users, and then score that code to determine if it is correct or not. We have leveraged cloud technologies to achieve this, and are able to support a wide variety of technologies to test, all built on a scalable solution that runs code in a fresh environment within milliseconds. Specific technologies include Docker, Kubernetes, MongoDB and NodeJS.

In addition to the ability to run code, we also have tools built to measure its performance, and a custom built solution for determining if code solutions are plagiarized.

Through the Qualified platform, and soon Codewars, we will be offering the ability to rate code for quality, so that it is not just checked for quantitative correctness, but also qualitatively. These ratings are tied into an advanced “signals” system which allows us to map user performance to our competency framework. Through this technology and community crowdsourcing, we are building out machine learning models which will eventually be able to learn what is a good or bad solution, and to be able to provide guidance to learners about how they can improve their code to be professional quality. The AI we build around this will allow us to scale our service so that only a small number of community mentors need to rate a subset of coding solutions, while being able to serve millions of users.

With our community built, our core technology well tested, and our competency framework ready, we are now focusing on our pathways product. Over the next year we intend to build the features needed so that our community can target specific career goals and learning objectives. We will build tools that make it possible for professionals in the community to create pathway focused content that aspiring professionals can challenge themselves on. We will focus on establishing certifications as part of pathway success, and partnerships which help us target communities who need pathways the most.

Over the next 5 years we will focus on bridging Codewars pathways with customers of our sister product, Qualified.io. This will allow us to not only mentor aspiring and junior developers, but also place them within companies. We will partner with bootcamps and other organizations to help companies fulfill their diversity hiring targets. We already have started to lay the groundwork to this via select partnerships, where we will help companies that are looking to place junior software developers from non-traditional educational backgrounds.

Financial barriers include the ability to hire enough resources to complete the task. Marketing costs need to be considered. Codewars has grown organically to this point and reached a number of underserved communities already, however without the resources or network to tap more directly into these communities, we risk not being able to serve them as well as we could otherwise.

Technical barriers include leveraging technology built for Qualified.io within the Codewars community to implement certifications and activities tied to pathways. Social and gamification features will need to be further developed to integrate pathways into the existing community. We will focus on behavioral techniques to encourage users to continue their growth down a pathway in order to reach their career objectives.

Cultural barriers include expanding our relatively small team while also continuing to target our own diversity objectives.

Market barriers include our ability to validate our platforms efficacy, and determine how much reach our product is able to ultimately make when targeting specific communities.

Resource needs can be limited if we are able to harness the power of our dedicated community and volunteers. Codewars is already moderated by volunteers on a daily basis. We have already built tools and continue to improve them. Like Wikipedia, Codewars is powered by volunteers.

Marketing and exposure can be solved by partnering with groups who already attempt to serve specific communities. We already have many active groups on Codewars utilizing it to teach underserved communities, we intend to continue to further build relationships with organizations who have similar goals.

As we look to expand our software development team, our own Qualified.io platform can be used to conduct unbiased performance assessments on candidates, and we can tap into the Codewars community to target users based on their performance, not on their pedigree.

Our plan for validation of efficacy is two fold. First, we have our Qualified platform which is currently used by hundreds of employers to hire. We can channel pathway graduates to these employers and measure their success. Additional, validation pilots such as the one being offered by the Workforce Board would allow us accelerate our learnings.

Between our two co-founders, one went to community college and the other is Indian-American. As a whole, our company has a variety of ethnic backgrounds, along with employees who were raised with differing socioeconomic statuses. Because of this, our team has first-hand experience and an understanding of the difficulties many face when breaking into the developer world. This unique view from within our company gives us insight into working with minorities to help them succeed in finding and securing a developer role. We have also proved the efficacy of our platform by hiring developers from within our Codewars community.

Codewars partners with the following organizations who utilize the core assessment platform in either a hiring or educational capacity:

• Apple
• Zoom
• Facebook
• General Electric
• 2U / Trilogy
• Chegg / Thinkful
• The Girl Code
• Code 2040

The hiring use-case involves using our assessment technology to build and implement a pre-screen, mid-round, or final round assessment. Educational use-cases revolve around using the assessment technology as a: curriculum accompaniment, admissions screen, formative diagnostic, and test for job-readiness.

Codewars is a free community. Operational costs are covered by Qualified.io, an enterprise assessment platform used by companies and educational companies to assess developers for hiring and educational purposes. Qualified.io operates on a subscription basis. 

In the future, we plan to employers access to talent pools through a subscription or per-job-req model.

As Codewars is committed to democratizing access to relevant technology education and skills training to as many underrepresented communities as possible, we strived to keep this platform free.

To achieve financial sustainability, we initially achieved this through grants and investments. Then, we built a commercial-focused software platform called Qualified, which leverages different proprietary aspects from Codewars, such as the programming challenges and coding environment infrastructure.

Qualified helps organizations hire, educate, and objectively assess engineering talent by enabling them to test engineers with skill assessments (from our extensive library of ready-made challenges), or teach engineers through an interactive coding environment.

To receive access to Qualified, organizations pay for the quantity of assessments they plan to run through Qualified as well as the number of managers and reviewer user accounts they require.

Our three primary types of customers are the following:

• Any Organization with Developers - Our platform helps them interview and assess new potential hires, educate their team, and/or certify their engineers objectively at scale.

• Education-based Companies – For organizations that are focused on software engineering education, Qualified helps them teach and test their students objectively and efficiently with Real-Time Pair-Programming and tests. Our comprehensive library of programming challenges eliminate the need to develop tests as well.

• Certification-based Companies – For certification organizations, we provide a standardized way to certify software engineering talent through our assessment library. They can deploy the same set of tests across individuals at scale and efficiently certify those that meet their requirements.

Grants and Investments

- Kapor
- Social Capital
- Learn Capital
- Cornerstone OnDemand Inc.
- Dalus Capital
- Hinge Capital
- Brian Lee (Legalzoom, Honest Company)
- Village Capital
- Michelson 20MM Foundation

Revenue Source Breakdown

- Companies with Developers – 50%
- Education Companies – 40%
- Certification Companies – 10%

For 2021, one of our goals is to develop more personalized, career-specific learning pathways from our comprehensive library of assessments (katas) for our Codewars community. This will enable them to focus on completing specific challenges and learning objectives that directly relate to the specific roles that they aspire to attain. In doing so, this should significantly improve employment outcomes for our community.

Our current financial sustainability strategy allows us to maintain Codewars as-is, however, with our 2021 goal in mind, we expect the following increases in our 2021 expenses:

- Payroll – An additional $200,000 to $300 000 – This is devoted to hiring additional software development and UX/UI expertise to enhance our pathways and learning objectives features. We also plan to bolster our kata library so that our community can continue to acquire all the necessary skills they need to get hired.

- Hosting and Technological Infrastructure – Estimated to be $10,000+ – We expect our organic growth to persist as we continue to enhance the learning and career placement features. Inevitably, our costs to maintain and host our software platform will increase as we serve more users.

- Advertising and Promotion – An additional $10,000 to $20,000+ – Although we focused on organic growth in the past and our user base has quickly grown through word of mouth due to our focus on keeping our learning platform free, we hope to reach more underserved communities who may not have heard of us yet with this budget.

While the grant money itself would help us reach our resource goals, the ability to tap into MIT and challenge partners networks would be an even larger benefit. In addition, the validation pilot program would allow us to validate and measure our effectiveness in ways that we currently are not able to.

We would like to work with schools and other learning organizations who focus on software development, especially those targeting underserved communities. Any partner which offers a solution to a learning pathway related to software development would be a good fit for our model, where we are able to acquire and retain interested learners and then focus them down paths which may involve their services.

Codewars to date has never marketed its platform and yet still has grown organically to 2 million users. With the help of marketing and additional exposure we should be able to reach deeper into underserved communities to give them a chance at building a career they may have not realized would be a possibility for them.