AutoGrade

Our venture automates the grading process for free response and long-form mathematics questions in order to save teachers time.

Middle and High School teachers lose 8+ hours to grading student responses on exams every week. We automate the grading process so teachers get that time back. They are no longer frustrated doing time-consuming, rote, and menial work that doesn’t even contribute to student learning. The long-form and free-response grading solution that we are developing will allow for all questions to be graded automatically. Because the only automated grading teachers can currently perform is on short-answer or multiple choice questions (which are the worst assessment formats because of the inability to assess higher-level thinking and known gender biases). Our solution uses a combination of Computer Vision and Optical Character Recognition machine learning methods to understand and interpret handwritten formulas. We then utilize math-based Natural Language Processing (NLP) techniques to process full step-by-step mathematical solutions, identify errors, and provide feedback. The teacher uploads student tests along with a rubric of teacher solutions for the test, and then the tool grades each question according to the provided teacher rubric.

The problem is that teachers spend a large portion of their working hours grading student work and providing feedback. The average US high school teacher spends 8 hours per week grading, which involves extra work outside of school hours and takes away from the time they have to help students learn. There are 1.8 million high school teachers in the US, meaning that 216 million dollars of taxpayer money is spent each week on grading. Automating grading would drastically reduce the workload placed on teachers and would give them back their time to spend helping students learn. Even more important, teachers will be happier, less-stressed individuals if they’re freed from doing such a rote and menial task.

Our solution serves teachers who have to grade student work (at this stage we are focusing on high school & middle school mathematics teachers). Currently teachers have to spend time each week grading student work in addition to time spent planning for lessons and actually teaching students. To do this work, teachers often have to work outside of school hours. Our solution will help teachers by taking the burden of grading off of them, allowing them to have more time to focus on planning and teaching their students.

We have conducted primary market research (PMR) interviews with numerous teachers to understand their problems and how we can help address them. We have also met with other individuals working in education (for example, managers at Kumon locations, which provide after-school educational services). We have spoken with 15+ Researchers in the area of Grading Solutions and Industry Engineers of EdTech to understand the challenges in development and deployment of tools for teachers. We plan to beta test our solution with math teachers in the coming weeks as we finish developing our MVP.

To this date we have conducted primary market research interviews and we have built out a prototype solution that can grade algebra 1 and pre-algebra questions. We hope to have this prototype completed by the end of January so we can begin getting feedback from users on our design.

Our solution uses a combination of Computer Vision (CV) and Optical Character Recognition (OCR) machine learning methods to understand and interpret handwritten formulas. Object Detection methods are used to process tests and identify questions and individual components of student solutions. Optical Character Recognition techniques are then used to parse individual solution lines into a textual representation. Formula processing libraries are used to convert the textual representations into symbolic representations. Math-based Natural Language Processing (NLP) techniques and symbolic evaluation toolkits are then used to process multi-step mathematical solutions, grade them, and provide feedback.

The problem is that teachers spend a large portion of their working hours grading student work and providing feedback. The average US high school teacher spends 8 hours per week grading, which involves extra work outside of school hours and takes away from the time they have to help students learn. There are 1.8 million high school teachers in the US, meaning that 216 million dollars of taxpayer money is spent each week on grading. Automating grading would drastically reduce the workload placed on teachers and would give them back their time to spend helping students learn. Even more important, teachers will be happier, less-stressed individuals if they’re freed from doing such a rote and menial task.

Impact Goals:

• Help reduce grading for 10,000 high school teachers, freeing them to spend their time on preparing lessons and teaching students.
• Reduce the grading time for each of these instructors by at least 90%.

We plan on reaching these goals by (in addition to rapidly improving the accuracy of our backend AI model) having an intuitive frontend user-workflow for students checking on feedback & teachers reviewing questions & class performance (UX-optimized frontend). 

We plan to measure our progress in the next year through:

• The number of teachers we have reached

• The reduction in grading hours per teacher

• The number of subjects that we offer the solution for (eg. pre-algebra, algebra 1, etc.)

Long-term, we hope to measure our impact on student’s learning through the following additional indicators:

• School completion rates

• Percentage of the population achieving literacy and numeracy

The main barriers we anticipate are as follows:

• Limited funding for new technologies in education and bureaucracy involved with approving new technologies for schools and with getting them integrated. We plan to initially provide our base platform for free to teachers to help overcome this hurdle.

• Challenges with student data privacy, for which we will have to develop robust techniques for anonymizing student data and for which we will likely need to get approval from different educational institutions.

• We will have to overcome inertia with teachers where they are resistant to changes with the tools that they use.

• Challenges with processing a wide range of test formats, for which we must build incredibly robust computer vision models.

• Challenges with processing questions with graphical components which will require more complex computer vision solutions rather than OCR/NLP.

Both me and my co-founder have extensive research and technical experience in ML. My cofounder, Coleman, has worked on research projects in machine learning spanning speech recognition, natural language processing, keyword spotting, and computer vision, and I have worked on computer vision and optical character recognition projects. Our intense technical understanding in these areas will allow us to harness recent research advances in order to develop novel technical solutions.

Both of us also went through the public school system; I attended public schools in Fremont, California, and he attended public school in Canada. We are both incredibly passionate about improving public education. I led science and math tutoring programs throughout high school and volunteered as a teacher in MIT’s ESP program for teaching middle & high school students. Coleman has worked and volunteered extensively with students; he has been a teaching assistant for six courses, provided tutoring and math help services for both high school and university students, and worked with math outreach programs in middle schools in Canada.