Introducing Rubrics for Assessment Plans! Review assessment plans in the same system where they’re entered and stored. Supply feedback in a standardized and meaningful way by reviewing against a rubric.
Enhancements to Watermark Planning & Self-Study
(NEW!) Review Assessment Plans Using a Rubric
Currently schools must rely on a combination of disparate tools, from spreadsheets to word documents, from outside Planning & Self-Study to accomplish Assessment Plan Review. With our new solution, users can conduct these reviews in a more convenient and concise manner, done within the same system in which they are completing assessment plans. This allows the institution to see how their program and non-academic units are doing in their assessment process and provides an outlet for supplying feedback in a standardized and meaningful way by reviewing an assessment plan using a rubric.
Rubrics for reviewing assessment plans can be created by the administrator. There are up to five proficiency levels and multiple criteria for evaluation, as well as a point value and a description for each criterion and proficiency level.
Create a rubric and establish a review process by navigating to the Plans area on the left-hand side and open the assessment plan for which you wish to establish a review. Please note this is only applicable for program-type assessment plans today. Click the program-type assessment plan for which you wish to establish a review process to which you will see the Rubrics button where you can construct a rubric.
View of creating a new Rubric for evaluating the program.
Creating a New Review
The administrator can create a review for an organization with a unique title, due date, rubric for evaluation and can then allocate it to multiple organizations. You can assign a specific reviewer to a specific organization or to multiple organizations, allowing for an easy routing experience.
This view is for creating a New Review.
View of Assigning a New Reviewer for the Review.
View of Selecting Organizations for Review.
Upon completion, the user is redirected to the Review Tab, showing reviews and their status, with a snack bar notification saying the Review was launched.
Email Notification for the Reviewer
After a reviewer is assigned and an assigned assessment plan is submitted, the reviewer will receive an email notification that the plan is ready for review. If the reviewer is assigned later in the process and multiple plans to which they are assigned have already been submitted, they will receive one email notification, rather than multiple emails cluttering their inbox at the same time. The reviewer will receive a reminder email to complete the review within 48 hours of the due date.
When a reviewer is assigned to a review for a program to be completed later, they receive an email that says a new assessment plan is ready for review.
When a reviewer is assigned organizations then they receive an email notification as shown above.
Reviewer receives this email notification when the review is due in 48 hours.
Toggle Buttons under Admin> Notifications for email notification.
As with previous email notifications, you may choose to turn these notifications off. The toggle button under Admin > Notifications now allows users to enable/disable email notifications.
Reviewer Receives the Review
When the work is ready to be reviewed, a Review card, named after the review, to access the submissions can be found under the Service and Committee Involvement section on the homepage.
When opening a submission, reviewers can see the work presented on the left and the criterion on the right. The “vertical view” vantage point of the rubric allows the submission and the individual criterion to be read together easily. Reviewers can select a proficiency level from the right side panel and then click show more to see the full description of that proficiency level.
When the Reviewer clicks Review and Submit, a full flyout with the rubric displayed in a more traditional format appears, with an error message if any criterion is left unanswered. When the toggle button is enabled, all of the criterion descriptions and the proficiency level are displayed. The reviewer can also leave comments on this page.
A Review card for the reviewer to finish the review is found under Service and Committee Involvement.
On clicking the Review card under Service and Committee Involvement opens a page, where the organizations under that review are listed.
Clicking the review link opens a page which shows the submission on the left and criterion to the right.
On clicking Review and Submit, reviewer is redirected to a full flyout where all the criterions are listed.
Contributor Receives the feedback
Once reviews are completed, contributors can easily access them through the Docs and Reports tab of the organization profile.
The feedback for plans still in progress appear in the In-Progress tab, while feedback for plans that have been completed appear in the Docs and Report tab.
Additionally, Admins are able to drill into individual programs or other unit reviews through the Organization Management area.
When the review results are opened, they can be downloaded as a PDF.
The feedback given by the reviewer is shown under Review Result.
Contributor receiving the feedback for a plan which is completed.
Contributor view of the feedback received from the reviewer.
Course Filter for Course Evaluations & Surveys Results Integration
We have provided Admins with the ability to filter the CES integration data by course or a set of courses. This means the integration now supports program or course-outcome level assessment. Before, users had to bring in all of the data for the selected questions of the survey. With the introduction of this feature, users can filter course evaluations. There is no maximum number of courses they can choose within a survey. The tools also continue to support institution-level learning outcomes assessment by allowing admins to not filter by courses.
NOTE: This feature is applicable only for Course Evaluation projects.
View of the ‘Filter by Course’ screen where users will be presented with filtering options. They can either filter the results by a particular course OR get the results for all courses.
View of the filter by course screen where the user has the ability to search & select the courses.
View of the filter results by course screen when the user is trying to navigate to get results for all courses after selecting a few courses.
If a user switches between the filter options, the system will notify that all existing course selections will be discarded, and that the user will have to make new selections.
View of the results card showing the count of courses.
The results card has been updated to accommodate the course information so that the user will get to know how many courses have been selected.
Users will be able to make changes in the aligned result selections. If the user deselects any selected course or they switch from All courses option to Filter by Course, they will get an alert to confirm if they would like to keep the existing alignment OR edit it.
As with the previous release, the leads and the admin that initiated the course evaluation data will receive an email notification when the results are ready to be viewed. Depending on the amount of data, results can take as little as 30 seconds and up to 15 or 20 minutes. In the meantime, the user does not have to wait on the measure page while the results are being retrieved.
Please note that the Refresh Aligned Results link that may appear when you open a plan does not apply to Course Evaluations & Surveys results as they refresh differently than the other integrations. The Refresh Aligned Results feature continues to work for all other integrations including Student Learning & Licensure and the LMS Integrations.
Refresh Aligned Results on the top right is applicable to LMS Integrations, SL&L, and Outcomes Assessment Projects.
Enhancements to Leader Insights
We’ve recently added some small but mighty new features to the new leader insights area, to equip you to answer more questions, drive progress, and navigate to deeper insights about assessment plans:
- Monitor Course Learning Outcome Definition
- Navigate to an Assessment Plan
- Email leads about progress
As a reminder, institution-level leads can access this area by selecting “Enter Institution” in the My Leadership area on their Planning & Self-Study home page, and then clicking the “View Insights” button. Administrators can access this by clicking on the top Institution profile link within Organization Management.
View Insights button on the Organization Management page for “Diana Prince Academy”, the top-level organization in the hierarchy.
Emerging for now within Planning & Self-Study, over time you can expect Leader Insights to grow in depth and breadth, and to expand to include insights from other Watermark products.
As a Watermark Labs feature, you’ll also see us refine the user experience and features in this area rapidly, so please don’t be alarmed if things evolve or look a little different when you next log in
Monitor Course Learning Outcome Definition
Institution leads and administrators can now understand the progress of courses adding learning outcomes and mapping them to institution learning outcomes.
The new Course metrics tab in the Mission & Outcomes Definition widget of Leader Insights.
Please note that drilldowns are available for organizations and programs, but are not currently available for course metrics.
Navigate to an Assessment Plan
Institution leads and administrators can now navigate to the plan for more insights from the Participation in Assessment Planning widget.
The new Assessment Plan Navigation link in the lower right corner of the Participation in Assessment Planning widget of Leader Insights.
Email leads about progress
Institution leads and administrators can now quickly email leads to celebrate progress or remind them to engage. A link to email leads for a given organization or program is available in all drilldowns from the Mission & Outcomes metrics, as well as the Participation in Assessment Planning widget.
Drill-down view of Participating Programs’ Progress by Status with the Email Leads link.
As a refresher, a Watermark Labs feature is functionality made available to everyone while we continue to improve upon it. It allows you to take advantage of these features today, provide feedback, with the asterisk that there may be some areas for improvement as we continue to develop the experience. The Watermark Labs status will only be applied for a limited time while we continue to actively update the work.