Studentsense

We are current graduate students and faculty at the University of Pittsburgh and the Learn Research and Development Center. Currently, Studentsense is an independent initiative but can leverage organizational support, such as IRB, legal, etc., from the university.

Studentsense is a technological tool that aims to create equitable learning opportunities for at-risk students by empowering educators to address the diverse educational, social, and emotional needs of students.

Students in secondary schools are dealing with complex academic and personal issues leading to disengagement from learning. Schools try to cope with student disengagement before it results in course failure, grade remediation, or even dropping out. Studentsense is our innovative solution that empowers educators and administrators to proactively support students’ academic, social, and emotional needs before they disengage from learning. We are working to integrate a conversational AI agent within traditional Learning Management Systems (LMS) in order to collect comprehensive student social and emotional learning data periodically alongside traditional early warning indicators like attendance, behavior, and class grades. Using this data, we are developing an iterative predictive model to create faster cycles of early warning indicators to prompt educators with actionable calls to interact and intervene with students at-risk.

Social and emotional learning (SEL) interventions have been found to have a strong relationship to engagement in learning and to improve personal and academic outcomes for students, regardless of race and SES, across time, place, and context (Allbright & Hough, 2020; Domitrovich et al., 2017; Durlak et al., 2011; Kim, Lim & An, 2022; Soland et al., 2019; Taylor et al., 2017).

While educational systems are addressing the social and emotional needs of students to a greater degree, research lacks an adaptive formative assessment model for collecting consistent, reliable SEL data from students during the school day, week, or year (Barnes, Domitrovich & Jones, 2023; Panayiotou, Humphrey & Wigelsworth, 2019). This prevents the ability to understand the current states of students and how they change over time as well as the effectiveness of SEL interventions. Studentsense aims to create better, timelier, and meaningful educational experiences and influence equitable student learning by tackling two high-leverage points in our theory of change. First, creating a formative assessment tool to collect and analyze student SEL data along with other indicators consistently. Second, Studentsense will maintain a predictive model from this data to initiate calls for educator interventions. The educators will engage in either SEL or academic interventions, improving overall student well-being, increasing academic engagement, raising class grades, and, ultimately, keeping students on-track. 

We imagine Studentsense can help transform schools into a place where every student's emotional and academic needs are not just recognized but met with effective, timely support from the educators students interact with every day.

Our formative assessment design represents a novel adaptive approach and is a direct response to calls from researchers to “embed implementation data collection, reflection, and adapted practice into program design and delivery in ways that create meaningful improvements” (Barnes, Domitrovich & Jones, 2023; p. 5). Studentsense SEL data collection will benefit from reliable measures to inform adapted practice. Our conversational AI agent will engage in repeated matrix sampling rotating through general research-based concepts, such as overall learning engagement and motivation, and domain-specific constructs, such as self-efficacy, self-management, social awareness (Meyer, Wang & Rice, 2018), teacher responsiveness, classroom as a caring community (Domitrovich et al., 2022), and growth mindset (Dweck & Yeager, 2019) that have demonstrated reliability in the range of α = .76 to α = .89 in large-scale studies. Our design efforts will engage in iterative design cycles to understand student and teacher responses when they use Studentsense. In this way, we will contribute to burgeoning work showing that within-student changes in SEL constructs are correlated with change in academic outcomes (e.g., Kanopka et al., 2020; Duckworth et al., 2010). But, unlike prior work where SEL is measured annually or semi-annually, our formative assessment system operates as a way to influence practice in real-time as well as a way to more frequently collect data to better ask how attention to students’ social-emotional well-being influences their learning. This will benefit the field by developing a longitudinal inter-individual dataset for further evaluation and model improvement.

This data collection will generate a rich time-relevant dataset of student SEL. From this dataset, Studentsense uses hierarchical predictive modeling to identify assistance-seeking behavior and generates calls for interventions from educators. Using this model, student cases will be analyzed to identify outliers in both status and change from the groups in which they are nested, such as district, school, grade, class, or teacher. Studentsense will then create calls for intervention from educators clearly communicating the student’s case and providing transparent documentation to administrators. In so doing, we streamline educators’ noticing, interventions, and responses to student needs in both academic and SEL domains.

In addition, by having student characteristics, traditional early warning data, and general and domain-specific measures, we can adjust the grain size of refinements to the model and create more targeted initiatives. This opens doors to new research questions such as, which students benefit, under what conditions, and how do we catalyze teacher expertise to promote equitable student achievement and overall well-being?

Our team offers a broad set of expertise supported by institutional assets of the University of Pittsburgh. Hanan Perlman, a Ph.D. student in the School of Education, focuses on transformative factors for improving school-based practices to increase equitable secondary graduation. He leads Studentsense driven by an interest in the social impact of effective technological developments in education. As a former EFL teacher in high-need schools, he became acutely aware of the needs of priority learners and the overwhelming demands on teachers. Arun Balajiee, a Ph.D. student in the School of Computing and Information, works with the Personalized Adaptive Web Systems Lab on the development of adaptive learning systems. He provides crucial product design and programming insights and skills for the quality development of Studentsense applications. Dr. Richard (Rip) Correnti offers expertise in innovative efforts to transform educator practice with associated changes in student learning demonstrated in RCTs (see e.g., recent coaching effects on reading comprehension (Correnti et al., 2021) and curriculum effects on writing (Crosson et al., 2023).

As our tool is currently in the ideation stage, we aim to develop our tool to a minimally viable product (MVP) for user testing, reliability in-use, data collection validity, and modeling accuracy. First, we are working on LMS integration and 1edtech interoperability from the outset of our tool. Second, we are leveraging existing partnerships of Pitt with over 50 districts in our geographical vicinity, currently in contact with two school districts. Finally, we are engaged with the University’s Innovation Institute which offers financial grants, provides access to a professional network, and supports student innovation in various fields.

Recent research on school-wide interventions to support students’ SEL showed positive effects on students’ disciplinary learning (Domitrovich et al., 2017). Reviews of evidence-based programs show that learning effects have been found across multiple studies examining SEL interventions (Kim, Lim & An, 2022; Stefan, Danila & Cristescu, 2022). At this early stage, however, interventions are applied at the school level and implementation studies focused on fidelity to a top-down approach. SEL Interventions might require different ways of thinking about implementation in more authentic applications (e.g., Wanless & Domitrovich, 2015). While evidence supports the effects of top-down approaches, the mechanism for how effects are manifest remain a mystery. Thus, a central goal for the field is producing evidence to uncover mechanistic understanding for an indirect effect of social-emotional competence on student learning (Panayiotou, Humphrey & Wigelsworth, 2019).  

At the same time, because our approach involves a data collection system built into students’ routines and connected to an LMS system – i.e., part of the school infrastructure - it provides novel opportunities to build mechanistic knowledge through longitudinal analysis. In addition, we plan to incorporate a pre-post design to set a clear baseline for each student and to evaluate both data, as well as understand which students benefit the most from intervention. 

As our tool is currently in the ideation stage, we aim to develop our tool to a stage of a minimal viable product. First, using an LMS-integrated conversational AI agent (Large Language Model) to elicit closed-ended responses to SEL domains. The student-facing inquiries will be low friction, 1-4 questions taking less than a minute, intended to increase responsiveness and reduce cognitive load and attrition from completion of the surveys. The questions will rotate in ML led matrix sampling through general research-based concepts, such as overall learning engagement and motivation, and core domain-specific constructs, such as self-efficacy, self-management, social awareness (Meyer, Wang & Rice, 2018), teacher responsiveness, classroom as a caring community (Domitrovich et al., 2022), and growth mindset (Dweck & Yeager, 2019). This data collection will generate a rich time-relevant dataset of student SEL.

In a second area of operation, Studentsense uses hierarchical predictive modeling supported by a whitebox ML algorithm to identify assistance-seeking behavior and calls for interventions from educators. This part of Studentsense will collect, store, and analyze the data to serve as a basis for a predictive model. Using this model, student cases will be analyzed to identify outliers in both status and change from the groups in which they are nested, such as district, school, grade, class, or teacher. Studentsense will then create calls for intervention from educators clearly communicating the student’s case and will provide transparent documentation to administrators.  This model would need a secure platform to store and run the model which can produce educator-facing calls for intervention. These interventions aim to promote student well-being, improve student academic achievement, and keep students on-track for class and grade completion.

Recent research shows positive learning effects across multiple studies examining SEL interventions (Kim, Lim & An, 2022; Stefan, Danila & Cristescu, 2022). At this early stage, however, interventions are applied at the school level and implementation studies focused on fidelity to a top-down approach. SEL Interventions might require different ways of thinking about implementation in more authentic applications (e.g., Wanless & Domitrovich, 2015). While evidence supports the effects of top-down approaches, the mechanism for how effects are manifest remain a mystery. Thus, a central goal for the field is producing evidence to uncover mechanistic understanding for an indirect effect of social-emotional competence on student learning (Panayiotou, Humphrey & Wigelsworth, 2019). 

At the same time, because our approach involves a data collection system built into students’ routines and connected to an LMS system – i.e., part of the school infrastructure - it provides novel opportunities to build mechanistic knowledge through longitudinal analysis. In addition, we plan to incorporate a pre-post design to set a clear baseline for each student and to evaluate both the overall impact of our intervention, as well as understand which students benefit the most from the intervention. 

Studentsense is designed to incorporate SEL data collection into existing LMS systems while leveraging conversational AI agents to elicit responses from students to research-based inquiries of SEL domains (Jagers et al., 2019). The student SEL responses are collected with early warning data from integrated LMS. Previous predictive data models have been able to identify students at risk (Chung & Lee, 2019; Sousa et al., 2021). Our structured predictive model will use this data to identify students who may need assistance. Studentsense will then generate calls for educator interventions to inform appropriate school staff. These timely, targeted, and action-oriented equitable interventions address the educational, social, and emotional needs of students within schools before they worsen.

Studentsense is designed to incorporate SEL data collection into existing LMS systems while leveraging conversational AI agents to elicit responses from students to research-based inquiries of SEL domains (Jagers et al., 2019). The student SEL responses are collected with early warning data from integrated LMS. Previous predictive data models have been able to identify students at risk (Chung & Lee, 2019; Sousa et al., 2021). Our structured predictive model will use this data to identify students who may need assistance. Studentsense will then generate calls for educator interventions to inform appropriate school staff. These timely, targeted, and action-oriented equitable interventions address the educational, social, and emotional needs of students within schools before they worsen.

2 full - time doctoral students and 1 faculty member.

In order to construct our MVP and engage in both areas of operation, we plan to concentrate our efforts in four performance areas. First, developing the conversational AI agent on the OpenAI platform to integrate with existing 1ed tech interoperable LMS systems. Second, developing a research based SEL data structure to collect student responses.  LMS providers, schools that will use our tool, and district offices of information and technology to ensure technological support and access to student data. Fourth, developing an accurate white box predictive model for needed educator interventions. This model would need a secure platform to store and run the model which can produce educator facing calls for intervention. These interventions aim to promote student well-being, improve student academic achievement, and keep students on-track for class and grade completion. 

As our tool is currently in the ideation stage, we aim to develop our tool to a stage of a minimal viable product (MVP) for user testing, reliability in integration and use, validity in collection of SEL data, and accuracy of predictive modeling for the 2024-2025 school year. At the MVP stage, we are focused on high school students and their teachers. Once this stage is achieved, we anticipate multiple avenues for expansion of Studentsense. First and foremost, by planning for LMS integration and 1edtech interoperability from the outset our tool will be able to scale relatively seamlessly to additional grade levels, schools, and districts already using compatible LMS. Second, we plan to leverage existing partnerships of the LRDC and University of Pittsburgh with over 50 districts and hundreds of schools in our geographical vicinity to support the implementation and development of Studentsense. Additionally, we believe that Studentsense can be successfully integrated into middle schools, merge data from school level transitions, and incorporate input from school staff, such as guidance counselors or administrators, in addition to input from parents. Fourth, we hope to incorporate additional features of the 1edtech certification to leverage additional user data and features of LMS to improve Studentsense. Finally, the University of Pittsburgh’s Innovation Institute offers financial grants, provides access to a professional network, and supports student innovation in a variety of fields.  

As our tool is currently in the ideation stage, we aim to develop our tool to a stage of a minimal viable product. With the support of the  Award, we will develop Studentsense MVP for user testing, reliability in integration and use, validity in collection of SEL data, and accuracy of predictive modeling. Studentsense has two main areas of operation. First, using an LMS integrated conversational AI agent to elicit closed-ended responses to SEL domains. The student facing inquiries will be low friction, 1-4 questions taking less than a minute, intended to increase responsiveness and reduce cognitive load and attrition from completion of the surveys. The questions will rotate through general research-based concepts, such as overall learning engagement and motivation, and core domain-specific constructs, such as self-efficacy, self-management, social awareness (Meyer, Wang & Rice, 2018), teacher responsiveness, classroom as a caring community (Domitrovich et al., 2022), and growth mindset (Dweck & Yeager, 2019). >span class="NormalTextRun SCXW45871272 BCX2">ill generate a rich time-relevant dataset of student SEL.