Comparative Data Report

How is this report different from existing reports?

• Flexibility
  • The new Comparative Data Report (CDR) allows you to compare assessment data across both Standards and Rubrics within the same report output.
  • You can get the data you need for accreditation reports by selecting from a variety of filters.
  • You can select from an extensive range of data columns.
• Time Savings
  • The flexibility and customizable options of the Comparative Data Report are designed to reduce the time you spend developing your accreditation report.
• Measures
  • You have the option to add Measures as additional data columns in your report output. Some of these Measures include **Student Count**, **Assessment Count** (total number of assessments), **Rubric or Element Mean**, and **Standard Deviation**.

How to Build a Comparative Data Report

1. To build a Comparative Data Report, start by navigating to the **Reports** area. Click the **Create New Report** button and then select **Comparative Data Report**.

   

2. Begin by specifying your **Report Input**. After you provide a **Report Title**, you will need to select at least one filter. After selecting at least one filter, you will be able to select Rubrics that contain assessment data associated with your selected filter.

   

3. Selecting **Add Filter** will open the filter panel. Select a filter type and then the **Next** button. You will then be prompted to set the parameters for your filter. When you have specified the parameters for your selected filter, select the **Apply** button.

   

   • The filter types are consistent with the data present in your instance of Student Learning & Licensure. For example, if you want to include only a certain “Program” in your report, how your institution defines “Program” will determine which filter you use. Your institution may have associated “Program” with your hierarchy, in which case you’d select **Hierarchy Node** to filter by Program. Conversely, your institution may have opted to associate “Program” with Student Demographic data, in which case you would select **Academic Program** as the appropriate filter type.

   • Filters are added one at a time; you may add multiple filters by selecting the **Add Filter** button. You may view your selections for each filter by clicking on the caret to the left of the filter name. You may also edit your choices by selecting the pencil icon.

     

     

4. Once you apply at least one filter, you have the option to select your rubric(s). Only rubrics with data associated with your selected filter(s) will be available in the **Rubrics** dropdown.
   • At least one filter is required before you can select rubrics.
   • From the **Rubrics** dropdown, you can search for rubrics.

   • You must select at least one rubric, and you may use up to 10 rubrics in your report.

     

5. Next, select your **Report Output**. Report Output includes options to help you customize how your report appears. The illustration below indicates how **Data Columns**, **Measures**, and the **Aggregated Row** will be organized in your report. Data Columns are on the left and Measures are included as columns on the right of your report.

   

   • Select **Add Data Columns** to include specific columns in your report.

   • You can select up to 6 Data Columns to include. Data columns will display the values that match your filters and selected rubrics.

     

6. Data is disaggregated based on the order of your data columns. The first column is the most aggregated, and the last data column is the most disaggregated.
   1. **IMPORTANT:** The order of your data columns will impact how your data is displayed and disaggregated.
7. You can adjust the order of your columns by selecting the arrows to the right of the data column name.

8. To add or remove columns, select **Edit Data Columns**.

   

9. After adding your Data Columns, we recommend adding one or more **Measures** and/or an **Aggregated Row**.

   

Measures display as additional data columns placed to the furthest right of your generated report.

1. **Student Count:** the unique number of students that completed an assessment.
2. **Assessment Count:** can be thought of as the number of assessments completed, which can be higher than the Student Count if students were assessed more than once.
3. **Rubric or Element Mean:**
   1. If you have selected ‘Rubric Element’ as a Data Column, the mean displayed will be the mean score for the specific Element. If you do not include ‘Rubric Element’ as a Data Column, the mean will be calculated as the average of the rubric scores across all assessments included in that particular row.
      1. If your row only contains a single assessment for one student, the mean displayed is the total score of the rubric.
4. **Standard Deviation:** the standard deviation of scores across the assessments included in each particular row.
5. **Element Performance Level Range:** the range of performance levels assessed for your dataset.
6. **Element Point Range:** the range of element points assessed for the filtered dataset.
7. **Rubric Performance Level Range:** the range of performance levels for each rubric assessed for your dataset.
8. **Rubric Point Range:** the range of overall points given for each assessed rubric in your dataset.
9. If you choose to toggle **Aggregated Row** to on, your generated report will include totals for any additional ‘Measures’ you selected.
10. When you have made all of your selections, select **Run Report** to generate your report.
11. From your generated report, you have several options:
    1. **Save:** allows you to save a generated report or save changes to an existing report.
    2. **Save As:** allows you to save a generated report with a different title, which is useful for using an existing report as a template.
    3. **Export:**
       1. **Aggregated CSV:** exports your report in CSV format with only the data columns and measures you’ve selected for the filtered dataset.

       2. **Disaggregated CSV:** exports your report in CSV format with any additional measures you’ve selected and ALL data columns associated with the filtered dataset.

          

Assessment Count Drill Down Option

Whenever you generate a report and include **Assessment Count** as one of your **Measures**, you have the option to select any of the values in the Assessment Count column to open the Assessment Result panel and view additional details about the assessments included in the count. Selecting an assessment’s link will open the assessment in a separate tab.

Report Example

To help you get started, we have generated an example of how you might build your Comparative Data Report to answer assessment questions. These examples will all build upon each other, demonstrating how you might alter your settings (Filters, Data Columns, etc.) to generate various reports.

Example Report Prompt: Using the CAEP Standards (e.g. R1.1), how did the Fall 2022, Elementary Education program students perform? Overall? By Gender? By Race?

Example Report - Sample Approach: How your institution utilizes SL&L will determine which filters you use. To address the prompt above, you may start with filtering by **Standard** and **Term**. To ensure the data is sourced from the correct program, you might want to identify how your institution defines “Program” (e.g., by **Hierarchy Node**, by **Major**, or the **Academic Program** field in your User Import file).

In this example, our students’ “Major” defines which “Program” they are associated with; specifically Elementary Education.

Next, select the rubrics that you used to assess student performance.

Then, for Data Columns, think about what you want to view on your report. You might choose to include **Performance Level** and then run the report by including both **Student Gender** and **Race Ethnicity**.

For example, to have the Measures columns (**Student Count**, **Assessment Count**, **Element Mean**, etc.) indicate the Performance Level of specific student demographic groups, and to have the report disaggregate the data by Performance Level, you would reorder the Data Columns so that **Performance Level** was the last Data Column listed. This ensures the generated report displays the data disaggregated by **Race/Ethnicity**, then **Student Gender**, and finally **Performance Level**.

Alternatively, you may choose to run two reports to see the breakdown by **Student Gender** separate from **Student Race/Ethnicity**. Run the report with one of these demographics, **Save**, then **Edit Report**, update the Data Columns with the second demographic, **Run Report**, and then **Save As** with a new title.

Report by Race/Ethnicity

Report by Gender

How you order your data columns will impact how your generated report appears and the values for any additional measures you might choose to include. If your generated report does not appear as expected, select **Edit Report** and try rearranging your selected data columns.