Comparative Data Report

How is this report different from existing reports?

• Flexibility
  • The new Comparative Data Report (CDR) allows you to compare assessment data across both Standards and Rubrics, within the same report output.
  • Get to the data you need for accreditation reports by selecting from a variety of filters.
  • Select from an extensive range of data columns.
• Time Savings
  • The flexibility and customizable options of the Comparative Data Report aims to reduce the amount of time you spend developing your accreditation report.
• Measures
  • You have the option to add Measures as additional data columns in your report output. Some of these Measures include Student Count, Assessment Count (total number of assessments), Rubric or Element Mean, and Standard Deviation.

How to Build a Comparative Data Report

1. To build a Comparative Data Report, start by navigating to the Reports area. Click the Create New Report button and then select Comparative Data Report.

2. Begin building a Comparative Data Report by specifying your Report Input. After you provide a Report Title, you will need to select at least one filter. After selecting at least one filter, you will be able to select Rubrics that contain assessment data associated with your selected filter. 

3. Selecting Add Filter, will open the filter panel. Select a filter type and select the Next button. You will then be prompted to set the parameters for your filter. When you’ve specified the parameters for your selected filter, select the Apply button. 

• Note, the filter types are consistent with the data that is present in your instance of Student Learning & Licensure. For example, let’s say you’d like to only have a certain “Program” included in your report. How your institution defines “Program” will determine what filter you would use. Your institution may have chosen to associate “Program” to your hierarchy, in which case, you’d select Hierarchy Node to filter by Program. Conversely, your institution may have opted to associate “Program” with Student Demographic data, in which case, you would select “Academic Program” as the appropriate filter type. 
• Filters are added one at a time; you may add multiple filters by selecting the Add Filter button. You may view your selections for each filter by clicking on the carat to the left of the filter name. You may also edit your choices by selecting the pencil icon.

4. Once you apply at least one filter, you have the option to select your rubric(s). Only rubrics with data associated with your selected filter(s) will be available in the Rubrics dropdown.
   • At least one filter is required before you can select rubrics.
   • From the Rubrics dropdown, you can search for rubrics.
   • You must select at least one rubric. You may use up to 10 rubrics in your report.

6. Next, select your Report Output. Report Output includes options to help you customize how your report appears. The illustration below indicates how Data Columns, Measures, and the Aggregated Row will be organized in your report. Data Columns are to the left and Measures are included as columns on the right of your report. 

• Select Add Data Columns to include specific columns in your report.
• Select up to 6 Data Columns to include in your report. Data columns will display the values that match your filters and selected rubrics. 

3. Data is disaggregated based on the order of your data columns. The first column is the most aggregated and the last data column is the most disaggregated. (See ‘Report Examples’ section)
   1. IMPORTANT: The order of your data columns will impact how your data is displayed and disaggregated.

1. You can adjust the order of your columns by selecting the arrows to the right of the data column name. 
2. To add or remove columns, select ‘Edit Data Columns’.

7. After adding your Data Columns, we recommend adding one or more Measures and/or an Aggregated Row.

Measures display as additional data columns placed to the furthest right of your generated report.  

1. Student Count: the unique number of students that completed an assessment.
2. Assessment Count: can be thought of as the number of assessments completed, which can be higher than the Student Count if students were assessed more than once.
3. Rubric or Element Mean
   1. If you have selected ‘Rubric Element’ as a Data Column for your report, the mean displayed in the report output will be the mean score for the specific Element. If you do not include “Rubric Element’ as a Data Column for your report, the mean will be calculated as the average of the rubric scores across all assessments included in that particular row.
      1. If your row only contains a single assessment for one student, the mean displayed is the total score of the rubric (average is total rubric score divided by one assessment).
4. Standard Deviation: the standard deviation of scores across the assessments included in each particular row.
5. Element Performance Level Range: the range of performance levels assessed for your dataset.
6. Element Point Range: the range of element points assessed for filtered dataset.
7. Rubric Performance Level Range: the range of performance levels for each rubric assessed for your dataset.
8. Rubric Point Range: the range of overall points given for each assessed rubric in your dataset.

8. If you choose to toggle Aggregated Row to on, your generated report will include totals for any additional ‘Measures’ you selected.
9. When you’ve made all of your selections, select ‘Run Report’ to generate your report.
10. From your generated report, you have several options: 
    1. Save: allows you to save a generated report. Also allows you to save changes to an existing report if you have made any edits.
    2. Save As: allows you to save a generated report with a different report title which is useful when you want to use an existing report as a starting point or as a template for a new report. Simply select an existing report, make your necessary changes, run the report, and select ‘Save As’ and provide a title for your new version.

3. Export
   1. Aggregated CSV: exports your report in CSV format with only the data columns and measures you’ve selected for the filtered dataset.
   2. Disaggregated CSV: exports your report in CSV format with any additional measures you’ve selected and ALL data columns, associated with the filtered dataset.

Assessment Count Drill Down Option

Whenever you generate a report and include ‘Assessment Count’ as one of your ‘Measures’, you have the option to select any of the values in the Assessment Count column to open the Assessment Result panel and view additional details about the assessments included in the count. 

Selecting an assessment’s link will open the assessment in a separate tab. 

Report Example

To help you get started, we have generated an example of how you might build your Comparative Data Report to answer assessment questions. These examples will all build upon each other, demonstrating how you might alter your settings (Filters, Data Columns, etc.) to generate various reports. 

Example Report Prompt: Using the CAEP Standards (e.g. R1.1), how did the Fall 2022, Elementary Education program students perform? Overall? By Gender? By Race? 

Example Report - Sample Approach: How your institution utilizes SL&L will determine which filters you use. Begin with identifying the specific assessment data you want to include in your report. To address the prompt above, you may start with filtering by Standard and Term. 

To ensure the data is sourced from the correct program, you might want to identify how your institution defines “Program”. Do you categorize a Program by ‘Hierarchy Node’,  by ‘Major’,  or do you utilize the ‘Academic Program’ field in your User Import file?*

*not applicable to SL&L institutions integrated with System Administration)

In this example, our students ‘Major’ defines which “Program” they are associated with; specifically Elementary Education. 

Next, select the rubrics that you used to assess student performance. 

Then, for Data Columns, think about what you want to view on your report. You might choose to include Performance Level and then run the report by including both Student Gender and Race Ethnicity. 

For example, here, we wanted the Measures columns (Student Count, Assessment Count, Element Mean etc.) to indicate the Performance Level of specific student demographic groups. We wanted the report to disaggregate the data by Performance Level. To achieve this, we reordered the Data Columns so that ‘Performance Level’ was the last Data Column listed ensuring our generated report displays the data disaggregated by Race/Ethnicity, then Student Gender, and finally Performance Level. 

Alternatively, you may choose to run two reports to see the breakdown by Student Gender separate from Student Race/Ethnicity. Run the report with one of these demographics > Save > Edit Report > Update Data Columns with the second demographic > Run Report > Save As - give the new report a title consistent with the change in demographic used.

Report by Race/Ethnicity

Report by Gender

How you order your data columns will impact how your generated report appears and values for any additional measures you might choose to include. If your generated report does not appear as expected, select ‘Edit Report’ and try rearranging your selected data columns.