The DRF Program settings define how participating individuals interact with the program's requirements. When creating a DRF Program, numerous preferences are set and participants are enrolled as Authors, Reviewers, and/or Evaluators to facilitate how work will be done, scored, and distributed.
Creating Your DRF Program
After creating a DRF Template, you will need to create a DRF Program to distribute the template to the appropriate participants. To create a new DRF Program, click TS Coordinator from the top navigation bar.
Click the DRF Programs link. (Please note, the link may also include TPA programs depending on your school's settings. These are specific to setting up a program for the edTPA).
Click the DRF Program button next to "Create a New."
The General Information will be the program name participants see on the home page. Enter a title and program description information (optional). After you have entered your course information, click the Create DRF button.
From the Setup tab, you will see an explanation of the different roles in the DRF Program.
Tab 1 - Setup
Reviewer Options: Reviewers can comment on work shared by authors in either a collaborative program or a DRF program. Authors can share folios, web pages, lessons or units for review. Typically, Reviewers provide formative assessment or comments. Reviewers differ from evaluators in that they do NOT have a formal evaluation role.
Make your selection by clicking the appropriate radio button.
Note: Enabling this feature does not make requesting feedback a requirement, but students will have the option to request feedback from you, other faculty, and/or their peers. If you select that Reviewers are not necessary, you can always change this option, if needed.
Evaluator Permissions:
| Options | Description |
|---|---|
| Evaluators can release evaluations and scores to Authors | This enables the Evaluator or All-Access Evaluator to release evaluations or scores to the Authors in this Program. If this option is disabled (does not allow release) then you need to assign an Evaluation Manager to handle the release of evaluations. |
| Can send work back for revision to Authors | This option enables an Evaluator or All-Access Evaluator the option to send work back to an Author for revision. |
| Can view other Evaluators' results | Enables an Evaluator or All-Access Evaluator to view any scores or evaluations that have been completed by other Evaluators. This permission does not enable the Evaluator or All-Access Evaluator to edit or cancel the scores. Note: the ability to view other evaluators' results extends to submissions requiring multiple evaluations; evaluators will be able to see each other’s' results (for the same submission) if this preference is enabled. |
| Can edit/cancel other Evaluators' results | Enables an Evaluator or All-Access Evaluator to view, edit, and/or cancel any scores or evaluations that have been completed by other Evaluators. Note: the ability to edit/cancel other evaluators' results does NOT apply to submissions |
| Can evaluate a re-submission from same Author | If work is sent back for revision, this option enables any Evaluator or All-Access Evaluator to score or evaluate the same work, by the same Author, for each requirement more than once. |
| Quick Score | Enables an Evaluator to submit the evaluation results in bulk rather than evaluate and submit their results for one Author at a time. Bulk Evaluation applies to the following evaluation methods:
When enabled, the QuickScore button becomes available in the Evaluation Grid, but only if the following conditions have been met:
mentioned above.
Evaluator. The Author work can be in any of the following statuses:
enabled) Progress by the same Evaluator) |
| Enable File Upload | When enabled, allows Evaluators to choose if they’d like to attach a file when either completing an evaluation or sending work back for revision. Evaluators can upload up to 3 files with a maximum file size of 5MB each. Evaluators can also choose whether or not these files are visible to the authors. |
Author Anonymity: This option encodes student names in the evaluation grid for Evaluators and Evaluation Managers. You can selectively make student names visible to specific Evaluators or Evaluation Managers. Anonymity is enabled for all students across all requirements in the DRF program.
Originality Detection: This feature enables your institution to seamlessly send Author work to Turnitin (an external vendor) for originality reporting (also referred to as “plagiarism detection”).
*Originality detection can only be enabled if the Turnitin integration has been enabled for the organization.
When you use the checkbox to enable this preference for a DRF, a Turnitin Originality Report is generated for all Author work that is submitted in the form of attachments, text areas, and forms.
Valid attachment types include MS Word, Word Perfect, PostScript, Adobe PDF, HTML, Rich Text Format (RTF), and plain text.
After the work has been submitted by the Author, Evaluators can view the Turnitin Originality Reports.
Turnitin notes that Originality Reports can take up to 24 hours to generate. However, Turnitin is usually able to process the files much faster than that, often within a few minutes after Author submission.
If you enable Originality Detection for this DRF, also use the radio buttons to indicate when/if Authors are allowed to view the Turnitin reports:
- Never allow Author to view the plagiarism report.
- Allow Author to view the plagiarism report after the work is uploaded to Taskstream.
- Allow Author to view the plagiarism report after the evaluation is released.
Versioning of Author Work:
When "versioning" is enabled for student and the work is sent back for revision, an Evaluator (including All Access Evaluators and Evaluation Managers) sending work back for revision has the option to create a non-editable copy of the current Author submission. The copy is displayed separately from the Author’s work area. The Evaluator can also decide if the Author should be allowed to import their previous work as a starting point for the new submission or whether they should start from scratch.
If you enable versioning of work, radio buttons are presented for you to select one how Evaluators will be able to view these copy "snapshots" of work:
- Allow Evaluators to view ALL previously saved versions while they are evaluating an Author’s subsequent submission.
- Only allow Evaluators to view the MOST RECENT work submission. Each time a re-submission occurs, Evaluators can only view the work submitted at that time and not previous work submission(s).
Next, you will need to set your permissions in the program. Note: Additional permissions can be added, if needed.
Setting Your Program Permissions:
Once you have made your selections, click the Save button to continue.
Tab 2 - DRF Template
Click the DRF Template tab select your course template that will be distributed to the participants in your program.
From the DRF Template tab, you can select the template you have started, by using the drop-down menu to select the category, My DRF Templates and click Go. Note: You may complete the step prior to finishing your DRF Template customizations.
Click the Select This Template button that corresponds with the appropriate template to make your selection.
Once you have selected a template, you will also indicate whether multiple Evaluators are needed or not needed. This refers to whether Author submissions require more than one evaluation before receiving a final score. Make your selection and click the Continue button.
Next, you will see the DRF Template structure including the category and requirement names. You will also find additional setup options in this tab as well:
- Number of Evaluators Per Submission
- Due Date
- Author Visibility
- Evaluator Visibility
Number of Evaluators Per Submission
This setting allows you to specify if certain requirements in the DRF need more than one evaluation per submission. Multiple evaluations will need to be reconciled so that the Author receives a single score. A DRF Program with multiple evaluations requires you to assign an Evaluation Manager with the Reconciliation permission to perform this function.
Due Date
If you wish to set a Due Date for a specific requirement in the DRF, click on the corresponding Edit button in that column.
On the Due Date Options screen, select Yes to set a due date for the category or requirement you selected on the DRF Template options screen.
Type the due date into the text field, or click on the calendar icon to the right of the text field to select the date.
*NOTE: Submission is due by the end (midnight) of the set due date of the Authors’ time zone.
If you want to allow Authors within the Program to submit work to this submission area after the due date, select Yes in response to the second question. Otherwise, select No in order to restrict submission after the due date has passed.
If you have chosen to prevent Authors from submitting work after the due date, you have the ability to grant exceptions to specific Authors within the Program. To allow certain Authors to submit work after the due date, first select the box next to Grant exception to submit work after the due date has passed to the following Author(s).
- Use the Reason/NOTE box to keep track of the reasons why you are granting an exception to the specified Author(s).
- Next, select the Authors to whom you would like to grant an exception in the Available Authors box. Click on the right arrow button between the All Enrolled Authors box and the Author(s) with Exceptions box to confirm your selections.
- If the name(s) do not appear in the Author(s) with Exceptions box, the Author(s) you selected from the All Enrolled Authors list will not be granted the exception.
If you want to remind Authors when the submission is due, select the box to Send a reminder to Authors who have not yet submitted work.
- (Optional) Click the "View all Authors who will receive this message" link to see a list of all the Authors who will see the reminder when it is sent.
- (Optional) You may edit the default subject and body text for the reminder you would like to send out. Click the Restore Default Message button at any time while editing the subject or body of the message if you want to clear your edits and return to the default message.
- The reminder will be sent through the Message Center within Taskstream. Unless the Author has turned off the external notification within Message Center, the message will also be forwarded to the external email address the Author used when registering with Taskstream.
- (Optional) To send a copy of the reminder to your own Message Center, select the box to "Send a Copy to My Message Center."
After making your selections, click the Apply Changes button to return to the main DRF Template area for the Program.
When the due date is saved, a calendar event is automatically created that will appear in the calendar of every Author enrolled in the Program.
Author Visibility of DRF
If the DRF Templates associated with the program has an evaluation method defined, you see an additional Author Visibility of DRF column which allows you to hide requirements or categories from selected Authors. By default, there is no restriction (set to All) for any area.
The Author Visibility of DRF column can display any of the following:
- "The Overall Folio Evaluation cannot be restricted" (ONLY if Overall Folio Evaluation is enabled)
- All - All Authors can see the selected requirement or category.
- Subset - There is at least one (1) Author restricted from access to the requirement or category.
- None - None of the Authors have access to the requirement or category.
- N/A - Admin category is not visible to any Author, therefore visibility restriction is not applicable in this case.
For a Program Manager to set the restriction, click Edit associated with that area to access the requirement restriction screen.
The legend above the Authors table displays a count of each work status in the table, as well as situations which prohibit restrictions.
Restricting access for all new Authors is a great way to hide an outdated requirement or a requirement that Authors should not have access to right away.
If all requirements within a category are set to be hidden for the Author, the category does not show up for the Author. This DOES NOT apply to enrollment from the TS DataExchange.
The Program Manager must select the checkboxes associated with the names of the enrolled Authors for whom they want to restrict access. The Can access be restricted? column statuses provide guidance for your selections. (When No Submission is Required, the work statuses are slightly different.)
- (Optional) To revert back to previously saved preferences, click Restore Last Saved Preferences.
- (Optional) Select All new Authors enrolled into this Program should NOT see this requirement to prohibit all future Authors enrolled in the Program to view requirements.
- (Optional) To navigate back to the DRF Template setup WITHOUT saving change, click << Cancel - Back to DRF Template Setup.
Click Apply Changes when finished.
Evaluator Visibility of DRF
If the DRF Template associated with the Program has an evaluation method defined, you see an additional Evaluator Visibility of DRF column which allows you to hide requirements or categories from selected Evaluators. By default, there is no restriction (set to All) for any area.
The Evaluator Visibility of DRF column can display any of the following:
- "The Overall Folio Evaluation cannot be restricted" (ONLY if Overall Folio Evaluation is enabled)
- All - All Evaluators can see the selected requirement or category.
- Subset - There is at least one (1) Evaluator restricted from access to the requirement or category.
- None - None of the Evaluators have access to the requirement or category.
- Requirement Level
For a Program Manager to set the restriction, click Edit associated with that area to access the requirement restriction screen.
The legend above the Evaluators table displays a count of each work status in the table, as well as situations which prohibit restrictions.
Restricting access for all new Evaluators is a great way to hide an outdated requirement.
If all requirements within a category are set to be hidden for the Evaluator, the category does not show up for the Evaluator. This DOES NOT apply to enrollment from the TS DataExchange.
The Program Manager must select the checkboxes associated with the names of the enrolled Evaluators for whom they want to restrict access. The Can access be restricted? column provides guidance for your selections.
- (Optional) To revert back to previously saved preferences, click Restore Last Saved Preferences.
- (Optional) To set all future Evaluators enrolled in the Program to not view the requirement, select All Subsequently Enrolled Evaluators should NOT see this requirement.
- (Optional) To navigate back to the DRF Template setup WITHOUT saving change, click <<Cancel - Back to DRF Template Setup.
Click Apply Changes when finished.
Tab 3 - Enrollment
Click the Enrollment tab to enroll participants into the program. From the Enrollment tab, you can enroll authors, reviewers and evaluators into your course. Authors (students) can be given a self-enrollment code if you choose not to enroll them manually. Reviewers and Evaluators are manually enrolled into programs.
If you’d like to create a self-enrollment code, click the Change button next to Allow Authors to Self-Enroll.
Select the radio button to change the status to Self-enrollment is ON. Enter a self-enrollment code or click Generate a Code.
When finished, click the Apply Changes button.
On the following screen, you will be able to access a Word document with instructions for using the code to self-enroll. This document can be distributed to Authors (students) who will be using the code to self- enroll. You can save a copy of this document to your computer.
To manually enroll a Reviewer or Evaluator for example, click the appropriate button.
This will open the Member Locator window.
You can enter an individual name in the Member Search field, or use the Basic Search area to find multiple names within the registration folders of your learning community. Check the boxes to select the appropriate individuals whom you wish to enroll, and click the Save and Close button.
Additionally, you can add a role to any program participant by clicking the appropriate letter next to the participant’s name. Once the letter is highlighted, the role is enabled for the program participant. For example, click the R icon next to the name of an enrolled participant, to add the Reviewer role to that participant.
Tab 4 - Grouping
Click the Grouping tab, to determine how Authors will interact with Reviewers (If Applicable) and Evaluators.
Reviewer Grouping
The Reviewer grouping preference appears ONLY if you selected Reviewers for this Program.
| Grouping Preference | Description |
|---|---|
| No Grouping Need for DRF | Default grouping; happens automatically if no other option is selected. Enables Authors to solicit comments from any Reviewer enrolled in the Program. |
| Multiple Groups for DRF | Enables a Manager to create groups and define from which Reviewers each Author can solicit comments. |
Evaluator Grouping
There are three (3) options, related to how and with whom, Authors submit work for evaluation.
| Grouping Preference | Description | Advantages | Considerations |
|---|---|---|---|
| No Grouping Needed for DRF | Authors submit work to a pool of all the Evaluators enrolled in the Program who have access to the category or requirement. Evaluators are anonymous to the Author. If there are multiple Evaluators in the Program, they need to decide who evaluates an Author, since any Evaluator can score any work submission. For this option, you do not need to create or manage groups. | Does not require any additional steps for the student. Does not require additional setup for the DRF Program Manager(s) | Evaluators may accidentally begin an evaluation for a student or assignment that they are not responsible for evaluating. |
| Multiple Groups for DRF | Authors submit work for assessment to the Evaluator(s), with whom the Author is grouped, who has access to the category or requirement. Evaluators are anonymous to the Author and only the Evaluator(s) grouped with a specific Author can score submitted work. If you select this option, it requires you to manually create and manage groups to pair Authors with Evaluators. | Ensures students’ submissions are filtered to the correct evaluators. Streamlines view for faculty so that they only see | Requires significant effort on the DRF Program Manager(s) to manually set up the groups. Potentially requires regular maintenance on behalf of the DRF Program Managers(s) if Author self-enrollment is being used. |
| No Manual Grouping, but Requires Authors to Select Evaluator | Authors select from a list of Evaluators, with access to the selected category or requirement, who they want to submit work for each part of a DRF. Authors must know which Evaluators they need to submit work and only the Evaluator that an Author selects for a specific submission can score the associated work. | Does not require additional setup for the DRF Program Manager(s). Enables faculty to have access only to students who have submitted to them. | Students may accidentally choose the incorrect Option does not work with no submission |
*Note- In grouping option 'No Manual Grouping, but Require Authors to Select Evaluator', the Author selects only ONE (1) Evaluator per submission. For additional Evaluators to access the submission (if multiple evaluations are required), you need to assign the All-Access Evaluator permission to any additional Evaluators who need to access and evaluate work.
Tab 5 - Resources
From the Resources tab, you have the option of attaching a web folio or web page you have created as a resource for program participants.
You can click to create a new web page or web folio, or select one you have already created from the appropriate pull down menu, and click the Select button. Program resources are optional. Note: This feature is optional.
Tab 6 - Status
From the Status tab, you can see any area of your setup that is incomplete. Remember that program resources are optional.
If you are ready to make your program active, click the Change Status to Active button. The program will now appear on the home page for all enrolled program participants.