Quill.org and our new tool, Quill Reading for Evidence

Tens of millions of low-income students struggle to write well, one of the biggest hurdles that low-income students face on their pathway to work, post-secondary training, and/or higher education. Writing and critical thinking skills can dramatically impact low-income student's ability to get a job, gain entry into a technical program, or matriculate and succeed at college. Yet, the National Assessment of Educational Progress reported that 88% of low-income students struggle with writing (2011). With nearly 27 million children - 38% of all US children - currently living in poverty (National Center for Children in Poverty), too many children fail to develop the skills necessary to write emails, answer questions on digital applications, and write essays.

Quill believes that preparing young people for post-secondary success, be it college or career-training, is the surest way to end cycles of poverty and help students access the agency and voice to successfully advocate for themselves and their communities.

However, low-income students face an incredible disparity in education quality and opportunities that are available as compared to their higher-income peers. African-American and Latino young people are over-represented in populations with limited access, opportunity, and networks for upward mobility.  

We know that these disparities have only worsened due to COVID, and teachers and administrators across the nation continue to face considerable restraints on their time and burnout from addressing challenges that COVID has generated and exacerbated.  

Quill.org offers resource strained teachers and schools an incredibly cost-effective solution (about $2.27 per student) that utilizes cutting-edge technology designed to improve low-income student's reading, writing, and critical thinking skills while saving valuable time and money.

Quill.org’s free online tools help low-income 3rd-12th grade students become strong writers and critical thinkers. Quill.org uses artificial intelligence to automatically grade and serve feedback on student writing, enabling students to revise their work and quickly improve their skills. Quill.org creates a personalized learning plan for each student by having students complete a diagnostic assessment that assesses a range of writing skills. The diagnostic report then recommends a set of high-impact activities for each student, and teachers can assign these recommended activities with one click. 

Our new tool, Quill Reading for Evidence is revolutionary in the field of educational technology, using artificial intelligence to provide students with immediate, real-time feedback on the evidence, logic, and syntax in their writing. To develop this tool, Quill had to learn how to develop, incorporate, and scale five different AI technologies developed by Google, Microsoft, and open source researchers. During this learning and evaluation process, we evaluated dozens of different solutions - a time-intensive research and development process that required more than $4 million in funding provided by philanthropic partners. 

This R&D process resulted in more than 100 different algorithms created by Quill that target a range of issues in student writing, including vague or inaccurate evidence, opinionated language, plagiarism, logic issues, spelling mistakes, and grammar errors. Moving forward, Quill’s product engineering team has learned how to quickly evaluate and incorporate new AI technology, enabling our team to rapidly build new AI-powered learning tools.

As part of the product development process, we conducted an initial usability study with Mathematica (funded by the Bill and Melinda Gates Foundation). That study found promising results, including: 

• 89% of students were able to complete the activity in under 20 minutes, indicating that teachers will be able to use Quill Reading for Evidence as a supplementary tool. 

• 100% of students independently navigated the Quill Reading for Evidence interface, and students found it to be intuitive and user-friendly.

• 77% of all students' revisions directly aligned with the provided feedback. 

• 79% of students reported that Quill Reading for Evidence is useful. 

Over the next two years, Quill will launch the tool and conduct cycles of learning engineering research with the College Board to determine the efficacy of the new learning tool. We will measure how accurate our feedback is (as compared to a senior educator’s feedback) and how this feedback impacts student writing. We will then measure our success by conducting a series of randomized control trial studies that examines the impact of our tool relative to market-leading literacy tools, with success defined as a larger increase in reading comprehension and writing skills. 

Quill’s nonprofit mission overall is to serve underserved students, and 67% of the schools using Quill are low-income, Title-1 eligible schools. Over the next two years, Quill expects to impact 3.5 million students attending low-income, Title-1 eligible schools. 

As highlighted above, a driving point of our work is addressing the incredible disparity in education quality and opportunities that are available to low-income students as compared to their higher-income peers, as well as the disparities that the African-American and Latino young people face. We know that these disparities have only worsened due to COVID, and teachers and administrators across the nation continue to face considerable restraints on their time and burnout from addressing challenges that COVID has generated and exacerbated. 

As a new tool in the suite of Quill.org's free tools, Quill Reading for Evidence offers an incredibly cost effective and targeted solution to help teachers at under-resourced schools improve writing instruction and support low-income student's development of the reading, writing, and critical thinking skills imperative to future success.  

By providing feedback on 500 million sentences per year, Quill will save educators four million hours per year in time saved grading homework. Since the average teacher’s salary is $25 per hour, the four million hours of time saved grading homework translates to $100 million in social value created through Quill’s feedback engine. We know that Quill Reading for Evidence will continue to offer a significant return on investment.

As a tool designed specifically for low-income students, Quill.org has worked and continues to work closely with educators at low-income schools to co-develop our tools, including Quill Reading for Evidence. This includes a network of lab schools in New York City with a population of low-income students who are predominantly African-American and/or Latino, where we have collaborated with educators to design and test new materials and an advisory network of researchers who help us identify the most effective, research-based strategies for helping students. 

By engaging these schools as our testing partners, and paying these schools to support their costs, we ensure that our software is developed with the specific needs and challenges of low-income middle school students of color, enabling these teachers to be successful with our tools to bolster students' reading, writing, and critical thinking skills.

Quill is positioned to scale, and we would appreciate peer guidance and expert advice as we launch our strategic plan to bring our tools to even more students.

Quill’s growth over the years has primarily been through word of mouth as teachers share our tools with other teachers; we do not have a marketing or brand development team, and our business development team consists of two employees. 

We have relied on the quality of our work - teachers are looking for high-quality, free tools, and when they find Quill, they often share it with others in their network. We measure how likely teachers are to promote Quill through our net promoter score, and our world-class +77 score shows that teachers are enthusiastic about using and sharing Quill.  To accelerate this process, we encourage teachers to share their experiences through blog posts, videos, and social media, and through these posts, we have reached tens of thousands of educators. 

As we continue to develop powerful new learning tools, and distribute those tools for free, a better understanding how to better promote our tools and bolstering our business development strategy would help us to reach even more students and educators. Our ability to scale our solutions - our growth and impact - will benefit immensely from this type of guidance. 

Quill is innovative because it changes the nature in which education technology approaches developing literacy skills and writing instruction. We have also been successful at demonstrating how ed tech tools can be integrated into classroom practice to save teachers time, improve instructional practice, and better support students' individual learning needs.

Educational technology frequently uses multiple-choice questions to help students develop literacy skills. With multiple-choice, students do not have the opportunity to construct their own ideas, and this limits engagement, understanding, and comprehension. At the moment, the majority of AI-powered writing tools focus on algorithmically grading essays, and they do not focus on providing the sentence-level writing strategies that build foundational writing skills. To provide students with robust feedback for students, the Quill team maps out hundreds of potential answers for each item through a combination of human judgment and machine algorithms. By mapping out these responses, Quill can then automatically serve immediate feedback on dozens of strong answers and thousands of potential errors, including logic, grammar, spelling, and punctuation errors. 

Quill.org also uses research-validated writing strategies to develop its learning tools. In 2016, we launched Quill Connect, a tool that builds basic writing skills through sentence combining, a strategy promoted through the Carnegie Corporation’s seminal Writing Next report. The report states, “Sentence combining involves teaching students to construct more complex and sophisticated sentences through exercises in which two or more basic sentences are combined into a single sentence. Teaching adolescents how to write increasingly complex sentences in this way enhances the quality of their writing (Graham & Perin, 2007, p. 18).” Quill is the first organization to build an online tool which provides instant feedback on open-ended sentence combining exercises by using AI to evaluate student results. 

During the 2017-2018 school year, Quill partnered with the University of Nebraska-Lincoln to conduct a large randomized control trial study with 1,141 students from 10 schools across four states. Dr. Hebert, the principal investigator of the research study, is a co-author of the Carnegie Corporation’s seminal Writing To Read the report. Dr. Hebert administered the Test of Written Language as a pre-intervention and post-intervention assessment to measure student writing scores. The study found that students in the treatment group had a statistically significant gain in their writing skills, with students using Quill improving their writing skills at 1.5x the rate of students not using Quill. We have attached the results from this study as an additional document. 

With these strong results in place, Quill has expanded its program by developing Quill Reading for Evidence, we are growing from a grammar tool into a literacy platform that helps students become analytical readers and writers. Over the next five years, Quill’s research & development team will build a series of five new tools that enable students to summarize evidence, develop thesis statements, and construct paragraphs. This sequence follows the pathway championed by Peg Tyre, a Quill.org board member, in a 2012 Atlantic piece titled The Writing Revolution. Peg joined the Quill board to help turn the research-validated strategies from her piece into free and open-source tools. 

Over the next 5 years, Quill will take the highest-quality open source curricula available and will embed that curricula into Quill, supercharging curricula with our AI-powered writing tools so that students can construct their own ideas in writing and receive immediate feedback designed to build their writing and thinking skills. Through an open-source approach of offering robust curricula materials and learning tools for free to all students and teachers, we expect to significantly impact 5 million low-income students per year by 2027, with a series of randomized control trial studies validating that these students are advancing their writing and thinking skills through our tools. 

For teachers to successfully incorporate Quill Reading for Evidence into their classrooms, they need content that is aligned to their core curriculum. To reach Quill’s long-term goal of impacting 5 million low-income students per year, Quill will partner with the top-rated open source curriculum providers on the market. We will embed their free curriculum into Quill and supercharge it with our AI-powered writing tools. By partnering with the best curriculum providers on the market, rather than developing our own competing curriculum, we can reach millions of additional low-income students. 

By 2024, Quill aims to have successfully launched partnerships with at least three national curriculum providers, with our first partner being the College Board. For example, Quill is now exploring a potential partnership with Match Education’s Fishtank Curriculum, one of the highest-rated curriculum offerings on EdReports.org. Quill could embed Match’s curriculum into our learning platform, disseminate it to the 100,000 teachers using Quill, and supercharge it by embedding the top-tier materials into our interactive learning tools so that students can receive real-time feedback. We will measure our success by requesting that EdReports evaluates and find that Quill’s implementation of the curriculum earns a “green-rated” score.

• Number of students who have successfully practiced and improved on one writing skill. 
• Number of students who have successfully practiced and improved on one writing skill who attend low-income, Title-1 eligible schools. 
• Number of students who have successfully completed the full intervention by writing and receiving feedback on 100 sentences. 
• Students who have successfully completed the full intervention by writing and receiving feedback on 100 sentences who attend low-income, Title-1 eligible schools. 
• Teachers Net Promoter Score
• Cost Effectiveness of the Intervention 
• Pre- and Post-Intervention skills, including:
  • Whether students improve their writing skills faster than students not using Quill.
  • % of time that Quill’s feedback is comparable to a senior educator’s feedback 
  • Whether students using Quill will more frequently include strong claims, evidence, and reasoning in their writing compared to students not using Quill based on a common rubric for evaluating writing. 
    

What instructional gaps are present in writing instruction today? 

Many students, but particularly students from low-income backgrounds, struggle to write clearly (NAEP, 2011). Low-income students often come from language-poor environments, and they enter into elementary school with a limited vocabulary. These students are often not instructed in the fundamentals of writing instruction, and they struggle to develop their writing abilities. The 2011 NAEP assessment has shown that 70% to 75% percent of students in 4th grade through 12th grade write poorly. According to the Nation’s Report Card, the latest year for which this data is available, only 1% of all 12th-graders nationwide could write a sophisticated, well-organized essay. Consequently, 40% of students who matriculate at community college require remedial writing instruction, and in the workforce, the inability of workers to write clearly is a persistent problem. An NBC article entitled, “Why Johnny Can’t Write, and Why Employers Are Mad” reports that the single most important attribute in hiring is written communication skills. In survey after survey, supervisors complain that employees lack the ability to write clear emails, whether that is to management, colleagues, or customers. 

What do literacy researchers recommend to address this problem? 

Low-income students need innovative writing solutions, and success is possible. Over the past 30 years, as knowledge-based work has come to dominate the economy, American high schools have raised achievement rates in mathematics by providing more extensive and higher-level instruction. High schools, however, are still graduating large numbers of students who do not have the writing skills needed to enter the 21st-century workforce. We know from the research how to remediate this problem. In 2007, Dr. Graham and Dr. Perin published the Writing Next report, a seminal publication sponsored by the Carnegie Corporation. Based on an analysis of 176 studies, the report recommends eleven strategies for building writing skills. The report findings “are based strictly on experimental and quasi-experimental research, as this is the only type of research that allows for rigorous comparison of effects across studies”. The recommended strategies include summarization, collaborative writing, and sentence combining. As the Writing Next report notes, some of these strategies have not been widely disseminated. Quill.org provides free and open-source tools to widely disseminate the research-validated writing strategies of Writing Next. 

What is Quill’s theory of action for building skills?

Students develop their writing skills on Quill by engaging in a cycle of feedback and revision. Once a student submits a sentence, the software serves a piece of feedback on a specific part of the sentence so that the student can revise his response. Quill’s feedback is designed to support the student while not revealing the correct answer, requiring the student to think about his work. Once the students revise their work, the system then serves another piece of feedback on another part of the sentence, and the student may revise his sentence up to five times in order to produce the strongest possible sentence. Through this feedback cycle, students progressively develop their skills. 

Quill addresses a number of syntactical issues, such as fragments, run-on sentences, adverb usage, adjective usage, compound sentences, complex sentences, parallel structure, and appositives. When a student is learning complex sentences, for example, the student would be tasked with joining together the following sentences: “The colonists thought the taxes were unfair. The colonists protested the taxes.” An emerging writer could write, “The colonists thought the taxes were unfair and they protested the taxes.” While using the word and is grammatically correct, the program differentiates the words and and since so that the student will specify the causal relationship between the two ideas, “Since the colonists thought the taxes were unfair, they protested them.” By teaching students how to show the causal link between the feeling of unfairness and the protesting, the student gain the ability to clearly express complex ideas in writing. 

How is Quill’s intervention a unique solution in the space? 

As educational software, Quill Connect is an exceptional solution for teaching sentence combining because students receive immediate feedback on their writing. At the moment, traditional sentence combining instruction requires teachers to manually grade student work, a time-intensive process that prevents students from receiving sufficient feedback and quickly understanding their errors. 

Quill is the first organization to build an online tool that provides instant feedback on sentence combining exercises. To provide this feedback, the Quill team maps out hundreds of potential answers for each item through a combination of human judgment and artificial intelligence algorithms. By mapping out these responses, Quill can then automatically serve immediate feedback on dozens of strong answers and thousands of potential errors, including logic errors, grammar errors, spelling errors, and punctuation errors. As a web-based tool, this feedback is served to hundreds of thousands of students, at virtually no per-unit cost for each additional student served. 

Why do literacy researchers promote sentence combining as an effective strategy? 

Dr. Bruce Saddler, a literacy researcher, writes, “Sentence combining has a well-established research base. In fact, over 85 studies conducted during the last 55 years have demonstrated that sentence combining is an effective method for helping students produce more syntactically complex sentences and may improve the overall quality of compositions .”

How does sentence combining improve writing skills? 

In 2005, Dr. Bruce Saddler conducted a sentence combining intervention and reports, “Forty-two students in the fourth grade received either sentence combining instruction or grammar instruction. Students were paired for instruction and received 30 lessons, 25-minutes in duration, three times a week for ten weeks outside of their regular classrooms. The results indicated that in comparison to peers receiving grammar instruction, students in the experimental treatment condition became more adept at combining simpler sentences together to create more complex sentences. In addition, for the experimental students, the sentence combining skills they learned transferred to a story writing task, resulting in improvements in writing quality as well as revising ability.”

What were the efficacy results from this intervention? 

In Dr. Saddler’s 2005 study, his study reports, “To assess the effects of the treatment, students were asked to write a story and then revise it immediately; sentence combining progress monitoring measures were given during the interventions; and administration of Form B of the TOWL test was administered to students after the instruction. The measures revealed a statistically significant effect of sentence combining instruction on all progress monitoring (effect size – 1.31) and standardized assessment (effect size = 0.81) measures. Participants in the sentence combining condition were twice as likely to combine two or more sentences into a semantically and syntactically correct single sentence than those students in the grammar instruction condition. Effects in the area of revision were more modest but moderate and still indicative of a higher level of achievement for the students in the sentence combining condition (effect size – 0.69). Additionally, for the students in the sentence combining condition, post-test story quality improved to some extent in this study (effect size – 0.64) whereas in the grammar condition it was unchanged.” 

Which literacy skills do students specifically improve in? 

Researchers have found that sentence combining expedites syntactic maturity and fluency, improves writing quality, increases the rate at which a student’s writing becomes more elaborated, and has lasting effects on all levels of student writing performance. In recent research, the strategy has also been used to improve writing outcomes for struggling students. Sentence combining instruction has even produced improvements in reading comprehension performance outcomes . 

What is sentence combining’s “theory of action” for improving writing skills? 

Sentence combining creates a positive impact on student writing due to three theoretical principles: (a) student writers need instruction in what a written sentence actually is, the limits of simple sentences, and the writer’s available syntactic choices; (b) the cognitive strain of writing is reduced once students become fluent with the process of combining sentences; and (c) gains in syntactic fluency lead to high quality writing . 

How does sentence combining teach the thinking skills of a mature writer? 

Dr. Saddler writes, “In my own writing, as I am putting thoughts down, I routinely look back over what I have written and change the arrangements of words within my sentences, or I may combine sentences or add words to an existing sentence. These are routine acts for any mature writer, but for many young writers (including writers with disabilities), making effective syntactic decisions is very difficult, as they may not know the choices that are available to them. Such writers may need to hear and read many sentence constructions that would not initially come to mind. However, even hearing and reading are not enough for many writers with disabilities, who will also need to physically manipulate the syntax.”

How do researchers describe the impact of sentence combining on student writing? 

Dr. Saddler writes, “Before I initiated sentence combining practice my students mainly saw the revision process as one of editing. They seemed to operate under a least effort strategy, meaning, they changed what was easiest to change. So they conducted “housekeeping” by fixing spelling, capitalization, formatting, and perhaps punctuation rather than engaging in what revising should have been, namely molding the sound of text to make a message clearer or providing an audience with what they need to know. But after sentence combining practice the number of revisions climbed. My students were changing words, adding phrases and clauses, re-working entire sentences – all of the behaviors we had been practicing. Clearly, in my mind, this meant that they were thinking about how to say things in a different way.” 

Quill.org uses research-validated writing strategies to develop its learning tools. In 2016, we launched Quill Connect, a tool that builds basic writing skills through sentence combining, a strategy promoted through the Carnegie Corporation’s seminal Writing Nextreport. The report states, “Sentence combining involves teaching students to construct more complex and sophisticated sentences through exercises in which two or more basic sentences are combined into a single sentence. Teaching adolescents how to write increasingly complex sentences in this way enhances the quality of their writing (Graham & Perin, 2007, p. 18).” Quill is the first organization to build an online tool which provides instant feedback on open-ended sentence combining exercises by using AI to evaluate student results. 

Quill Reading for Evidence, our new reading and writing tool, is the most complex learning tool that Quill has built to date, and it is a revolutionary tool in the field of educational technology. This tool uses cutting-edge artificial intelligence to provide students with immediate, real-time feedback on the evidence, logic, and syntax in their writing. To develop this tool, Quill had to learn how to develop, incorporate, and scale five different AI technologies developed by Google, Microsoft, and open source researchers. During this learning and evaluation process, we evaluated dozens of different solutions - a time-intensive research and development process that required more than $4 million in funding provided by philanthropic partners. This R&D process resulted in more than 100 different algorithms created by Quill that target a range of issues in student writing, including vague or inaccurate evidence, opinionated language, plagiarism, logic issues, spelling mistakes, and grammar errors. Moving forward, Quill’s product engineering team has learned how to quickly evaluate and incorporate new AI technology, enabling our team to rapidly build new AI-powered learning tools.

In 2021, Quill was awarded a grant by the Chan Zuckerberg Institute to participate in Promise54’s Diversity, Equity, & Inclusion (DEI) Accelerator, an eight-month intensive that brings together 40+ education organizations to engage in DEI action planning and capacity building in service of sustainable change. The program begins with Promise54’s extensive DEI survey, which provides organizations with in-depth information about their own organization and is benchmarked with data from over 20,000 individuals from 400 education organizations. As a result of this work, Quill has gained key insights and has developed a set of OKRs (objectives and key results) to implement these insights into our organization. Our objectives include the introduction of a DEI Leadership Steering Committee, revisions of our content and product development process to create more culturally responsive curriculum, new professional development offerings, and a new career pathways system.

Quill’s entire operating budget is covered through earned revenue and all philanthropic capital is currently allocated to new research & development initiatives; Quill does not require philanthropic capital to sustain its operations. Quill generates 66% of its income through earned revenue: 55% through publishers licensing our technology and 11% through schools and districts subscribed to Quill Premium. Quill’s philanthropic revenue is 34% of all revenue, and this funding enables us to develop powerful new tools that help us reach millions of additional students. 

School Revenue: Quill now has 500 schools paying for our premium service and 10,000 schools where teachers are using our free service. Over the coming years, Quill will secure paid partnerships with thousands of additional school districts to provide sustainable, long-term funding as we scale. Quill’s growth is driven by teachers - teachers are searching for high-quality free tools, and, when they find Quill, they often share it with others in their network. Once teachers are successful with our free service, administrators reach out to purchase our Premium services. To increase our school sales revenue, we will continue to invest in our Partnerships Team and Quill Premium offering by thoughtfully examining, experimenting, and iterating our customized instructional work, sales and customer service cycles, and data analytics offerings for schools. 

Publisher Revenue: Quill has been able to quickly scale its earned revenue by licensing our technology to educational publishers. These publishers, including the College Board, are eager to augment their own offerings with our robust content library and powerful technology. While Quill’s publisher revenue provides Quill with immediate capital to cover both operating costs and to invest in R&D, we recognize that there is a “concentration risk” of having a large portion of our revenue coming from a small number of publishers. As we scale, Quill plans to increase our district sales earned revenue from 8% to more than 50% of all of our earned revenue to ensure Quill can sustainability serve school districts for decades to come. 

Foundation Revenue: Quill has successfully attracted and sustained partnerships with major foundations due to the massive number of students that we have impacted as a result of our innovative learning tools and robust partnership engagements. We maintain a core group of philanthropic partners that have provided multiple-year and annual gifts, and who have served as trusted strategic advisors throughout the course of our work together. 

To accelerate our growth, we have built a pipeline of strong prospective partners for cultivation in the next two years of growth. Quill’s primary strategy for attracting philanthropic revenue is to provide our funders with a high social impact “return on investment.” We equip low-income students with better writing skills at a low cost per student impacted, and these costs will go down as we scale. By providing philanthropic partners with an attractive impact ROI and innovative product development, Quill will be able to attract and retain foundation partners dedicated to helping low-income students become stronger writers and critical thinkers.

As highlighted above, Quill does not require philanthropic capital to sustain its operations. Quill generates the majority of our income through earned revenue.

We currently have over 500 schools paying for our premium service and 10,000 schools where teachers are using our free service.We have been able to quickly scale earned revenue by licensing our technology to educational publishers, including a multi-million dollar contract with the College Board. We have also successfully attracted and sustained partnerships with major foundations like the Bill & Melinda Gates Foundation, the Booth Ferris Foundation, the Brothers Brook Foundation, the Chan Zuckerberg Initiative, the Lone Pine Foundation, the Louis Calder Foundation, and, most recently, a six-figure grant from the Overdeck Foundation.