Introduction
The feedback flow process for Program Review in Watermark Planning & Self-Study is a tool that can be used as an integral part of a college/university's quality assurance system. In this process, an institution is able to set up a multi-stage review sequence for their programs or non-academic units to submit their completed program review reports. In the feedback flow set up, admins are able to create feedback flows by defining stages and prompt questions for response at each stage. Next, the feedback flow can be assigned to all or a subset of programs in a program review cycle where reviewers are assigned to each stage so that these reviewers can provide their feedback. Reviewers can be assigned to a specific program in a stage or one reviewer can review all programs in a stage.
*Note: Creating a Feedback Flow is optional and is not needed to administer a Program Review.
Tools for Feedback Flow Process
The feedback flow tool allows the administrators to:
- Create a feedback flow process
- Assign feedback flow process to a Program in program review and align reviewers to each stage
- View the feedback provided by the reviewers for all the stages in the feedback flow
Creating a Feedback Flow (Video Tutorial)
You can watch the video below and follow the process, or you can follow the steps below.
Creating a Feedback Flow
When the program review administrator logs in, they see Program review in the side menu on the homepage. Once they are on the Program review page, they can create a Program review template or a program review cycle.
Under the Feedback and Approval tab, administrators can create new feedback flows by clicking on the “Create Feedback Flow” button. Here, they are able to create stages and add a feedback prompt question for each stage. A maximum of seven stages can be added in a feedback flow
- Feedback Flow Title: Here the administrators can enter the title for the feedback flow. The feedback flow can be used across program review cycles, so the title can be very generic. Each feedback flow must have a unique title. The feedback flow will be identified with this title while aligning it to a program in a program review in further steps.
-
Stage Title: Various stages through which a feedback flow passes can be created here. Users need to enter a title for each stage. The title should reflect who is reviewing in that stage, such Deans, Admin Review Stage, Committee review stage, etc. Please note that today each program or unit will only receive one review from one reviewer per stage. If you need to have more than review from the same group, you will need to have a stage for each review needed (e.g., committee member review 1 and committee member review 2). Alternatively, you could have those members compile one review and have one reviewer go into the system and enter that review for the stage. The title of a stage should be unique within a feedback flow.
-
Prompt: By clicking on the Add Prompt link, administrators can add a feedback prompt which acts as instructions or question for the reviewer of the respective stage
After creating the desired stages and adding prompts for each stage, the program review administrators can save the feedback flow by clicking on “Save Feedback Flow”. The saved feedback flows will be displayed under the “Feedback And Approval” tab.
*Note: An administrator can add a single prompt for each stage. If the reviewer needs to fill out a form or a rubric, they will need to fill it out outside of the tool and upload it as an attachment to their review.
Manage Feedback Flows
Clicking on the action menu button corresponding to each program review present under “Cycles” tab gives access to the “Manage Feedback Flows” area where the administrators can assign the feedback flows to the programs of a program review.
Once the administrators are on the “Manage Feedback Flow” page, they can start the process to assign feedback flow to the programs present in the program review. By clicking on the “Assign Feedback Flow” button, they will be taken to a two-step process.
Step 1: Assign Flow and Programs
On Step 1, users need to select a feedback flow from the drop down which they want to assign to the program(s) of the program review. All the feedback flows created under “Feedback And Approval” tab are listed in this drop down in alphabetical order.
Further, the user needs to select the program(s) to which the feedback flow needs to be assigned. Users have the ability to select a single program or multiple programs. All the programs that are included in the program review are listed here.
After filling all the information on Step 1, the user can proceed to Step 2 by clicking on the “Define Stage Details” button.
Step 2: Define Stage Details
On step 2, the administrators need to assign reviewers and a due date to each stage of the feedback flow.
There are two ways to assign reviews to each stage:
- With the toggle button “ON” for the option “Apply reviewer to all programs”, the user needs to assign a single reviewer and it will be assigned to all the programs for that particular stage.
- With the toggle button “OFF” for the option “Apply reviewer to all programs”, the user needs to assign a reviewer for each program under the particular stage. This gives the flexibility to assign a different reviewer to each program for that stage.
Administrators can assign reviewers by clicking on the “Assign Reviewer” link. This opens a modal window which lists all the users in the OA and administrators can select one from the list or search for a user. Only ONE reviewer can be selected for each stage/program. The reviewer must have an account created for them in the system.
After assigning the reviewers and due dates for each stage, the administrators can complete the feedback flow assignment process by clicking on the “Complete” button. The assigned feedback flows will be displayed on the “Manage Feedback Flow” page.
NOTE: Once the review for a program has started, the program cannot be removed from the feedback flow and cannot be added in a new feedback flow. Once the review is complete (all stages), the program can be assigned to another feedback flow.
Ability to Edit In-Use Feedback Flow
Administrators can edit the feedback flow templates to make corrections, like fixing a typo or clarifying instructional text, even after the flow is aligned to a program review. They can change the feedback flow title, stage title and prompts. However, they cannot add or remove a stage in a feedback flow that's in use. Any changes made in the feedback flow will be reflected in all instances where the feedback flow is used.
Option to Edit the Feedback Flow
Ability to Create a Copy of Feedback Flow
In case administrators need to make major changes in the feedback flow without affecting the existing program reviews where it's being used, they can create a copy of an existing feedback flow. All the stages and prompts are copied in the new feedback flow, which users can then edit further as needed.
Option to Copy the Feedback Flow
Reviewers’ Feedback
When the contributors have finished entering the data for the program review, they submit the program review report. Once the program review report is submitted, the reviewer is able to see the card of the program review for feedback under the Service and Committee Involvement section on the home page.
When the reviewer clicks on the link of the program review, they are taken to a page which lists the programs for which the reviewer needs to provide the feedback.
Once the first Reviewer either Saves or Submits the Feedback, the Program Review will lock and become non-editable to the Contributors who submitted it, until all stages of the Feedback Flow are complete.
After a review has been submitted for feedback, email notifications cannot be resent. Reviewers may login to complete the review via the link in the email received or via the link on their Planning & Self-Study homepage in the Service and Committee Involvement section. They will also receive a reminder email 48 hours prior to the feedback due date.
NOTE: If there is only 1 program review report that needs feedback, then the reviewer will land directly on the feedback page. This list of programs will NOT be displayed in case of only ONE program.
Clicking on the “View Report” button for any program lands the reviewer on the feedback page. In the left panel, the HTML report of the program review is displayed which shows the data entered by the contributors for various categories and sections along with evidence (if any) uploaded by the contributors.
In the right panel, the reviewers have a text box where they can add their feedback in reference to the program review report displayed in the left panel. The reviewers also have an option to upload files along with their feedback. A maximum of 5 files can be uploaded. Reviewers can expand the feedback panel in order to better view the text entered in the feedback text box.
Once the reviewers have entered their feedback, they can either save a draft in order to revisit again and complete the feedback. Or they can submit the feedback in order to submit their reviews. Once the feedback has been submitted, the reviewer will not be able to revisit it.
The feedback flow process follows a sequential order. First, the review comes to the reviewer of the first stage. As long as the reviewer of the first stage has not submitted the feedback, it does not move to the next stage. Once the first stage reviewer submits the feedback, the review moves to the next stage reviewers and so on. The feedback process is complete when the reviewer of the last stage submits the feedback.
Contributors’ Ability to View the Reviewers’ Feedback
Once the feedback process is complete, the contributors are able to view the feedback provided by the reviewers. When they click on the link of the feedback flow in the Org card on the home page, the HTML report of the program review is displayed in the left panel. In the right panel, stages of the feedback flow are displayed as accordions. The contributors can expand these and view the feedback provided by the reviewer.
Note: Once the feedback is reviewed and the contributor clicks Edit to begin editing (if needed), the feedback can no longer be viewed unless the Program Review report is submitted again. The contributor will need to download any attachments and screenshot/copy any feedback they want to keep available while editing the narrative.