Evaluating Student Risk

 

This article provides an overview of the Course Completion and Student Persistence predictive models and a recommendation for how to best utilize the risk indicators positioned throughout Student Success & Engagement or SS&E for evaluating the risk associated with students completing a course and the risk associated with students persisting to a future term.

All deployments of Watermark Student Success & Engagement come standard with a Course Completion model, however, in some cases your Institution might have additionally purchased a Student Persistence Model which will add the risk assessment of student persistence to your SS&E instance creating an additional risk indicator to a student’s record. This is explained in more detail below under the section Risk Calculation Display.

A predictive model is created by applying statistical methods to historical data at an Institution to derive a mathematical equation that produces a predicted probability of a desirable event occurring. Essentially, it is a process used to identify the historical characteristics and behaviors collectively demonstrated by a student in relation to a student's completion of a course or demonstrated persistence to a future term. The output from the equation is a Risk Score between 0 and 100 and can be interpreted as a probability of success. This score is then transformed into a Risk Level represented by the colors Red, Yellow or Green; demonstrating High, Medium or Low risk respectively. These Risk levels are determined using Risk Thresholds that are decided upon at completion of the model build in collaboration with your Institution and used to segment the Risk Score into the three colors represented. The example below is a case where the upper risk threshold is set at 80% while the lower risk threshold is set at 70% splitting the risk score range into the three risk levels of Low Risk (Green), Medium Risk (Yellow) and High Risk (Red).  

Ana1.png

Risk Types

Two different types of risk can be calculated and displayed within SS&E. The most common risk assessment, course completion risk, is displayed by default while term persistence risk can additionally be displayed upon request by the Institution. This risk is assigned to both new and returning students for the sub-population of students identified during the preliminary model review meeting held with your Institution. Often this is the 'general' population of students at your institution identified by removing non-degree seeking and high school student populations, however, in some cases your Institution may have opted to include alternative or multiple populations in your risk assessment strategy. Please refer to the model documentation provided to your institution at the time of your final model review or inquire with your SS&E Administrator at your Institution.

Course Completion Risk

In the case of course completion risk assessments, historical course section registration records are analyzed in order to produce a predicted probability that a registered course will in turn complete and result in a passing grade for a specific student. This predicted risk is assigned to each course section that a student may register for and is assigned at time of registration and again re-evaluated daily taking into consideration subsequently collected data for the student.

Term Persistence Risk

In the case of student persistence risk assessments, historical student information including but not limited to academic ability, ability to pay, geo-demographic characteristics, past institutional academic performance, etc... are analyzed to produce a predicted probability that a student having registered for at least one course will persist to the very next term. This predicted risk is assigned to each student at time of registration of their first course for the upcoming term and again re-evaluated daily taking into consideration subsequently collected data on the student.

Risk Indicator Display

The SS&E Risk Indicator is a symbol (e.g.  Ana2.png) used to represent Risk Level at both the Student and course level. Risk Level is represented by this indicator placed at the student level  (i.e. by the students name) on the Daily Activity feed and the Case Load navigator within the student profile screen and is meant to provide the user with a high level indication of the respective risk prompting the user to explore the Risk Score (See Risk Score section below) further if warranted (See Risk Score section below). A institution that has only implemented the base Course Completion model will have only one Risk Indicator visible on the student record, however, in cases where a Term Persistence model is also implemented two indicators will be visible. The Term persistence indicator will always be located to the left of the Course Completion indicator, however, you can hover over either indicator in order to identify, of the two, which is indicating which risk. An example of these indicators displayed on each student level record is shown below.

 Ana3.png

Additionally, a Course Completion Risk Indicator will also be placed on the Term tab within the student profile navigation panel. This will be the same indication of risk that was displayed on the student level record and is meant to provide further indication that one or multiple courses for the current term are at risk.

Ana4.png

It is important to understand that when predicting Student Persistence Risk with the term persistence model the prediction is for the student and explains their likelihood of persisting to the very next term. Thus the Risk Indicator is a direct reflection of that students risk to persist. The Course completion model, however, provides Course Completion Risk Scores for each course section a student may register for. Therefore, the Course Completion Risk Indicator displays Course Completion Aggregated Risk. This risk is essentially the highest risk level represented on any single course that has been registered for during the current term. The table below displays the definition of the risk level displayed for a student when Course Completion Aggregated Risk is being displayed within the Student Level Course Completion Risk Indicator.

Ana5.png

Ana6.png

Risk Indicators regardless of whether it is representing Course Completion Aggregated Risk or Student Persistence Risk can be used to filter students within the Student News Feed, Advanced Search or message view within SS&E. An example of the filtering used within the Activity feed is shown below.

 Ana7.png

Risk Score Display

While Risk Indicators can provide us with an indication of student risk, Risk Scores provide a more detailed visual of Student Persistence Risk and Course Completion Risk. The bar below can represent the Risk Score for student persistence when displayed within the Student profile header or for course completion when displayed within the course section title for each course listed within the Term tab.

 Ana8.png

The percentage listed (Risk Score) is the probability of the student persisting opt the next term or completing the respective course and the Risk Level is identified via the color of the bar. Notice the bar represents 100% when fill completely, however, the bar will be filled only to the level of the Risk Score demonstrated for each situation.

Clicking on the Risk Score will open up a new window providing further insights into the Success and Risk factors influencing the students likelihood to Persist or Complete a specific course.

Ana9.png

Configuration

Administrators can choose which roles within their institution or to even show model results at all. To Configure the model(s) at your institution visit Administration from your profile drop down and select Configuration within the Analytics section of your side navigator.

 Ana10.png

Risk Indicator Usage

In the scenario where only a Course Completion model is installed for the Institution, using the student level risk indicator will alert the student success team of a potential medium or high risk course that might exist on a students registered course load allowing for the team to identify students early during the registration period that might benefit from an adjusted course load. Additionally student’s identified during the initial few weeks of a term can benefit greatly from increased interaction, accountability and potential referrals to tutoring or other resources that can assist with the student’s likelihood to complete the course. Its recommended to focus first on student’s who are ‘Yellow’ and then students who are ‘Red’. By doing so, the student success team will assist in the improvement of the course completion rates for students at the Institution and in turn likely impact the long term success of students at the institution.

In the scenario where a Student Persistence model is installed for the Institution using the student level risk indicator will alert the student success team of a potential medium or high risk student that are at risk of not returning during the next term. Using both models in tandem provides an additional layer of insight into a student and their potential boulders presenting challenges with Persistence and / or Course Completion for a student.

 

 

 

Articles in this section

How to Contact Support
There are many ways to reach out! Click here for our support options.
Watermark Academy
Click to access the Watermark Academy for consultation, training, and implementation companion courses.
Customer Community
Can’t find the answer? Ask fellow users how they’re making the most of Watermark in our Community!