Reporting and Analysis

Updated
Watermark Support

LiveText Reports

Users can create comprehensive, clear reports in LiveText that visibly reflect the assessment process and assist in making meaningful improvements at the course, program and institutional levels. Individual and aggregated reports can be generated on student progress as well as on the meeting of outcomes and standards. A brief overview of the following reports will be provided in this section: the C1 Assessment Report, Curriculum Mapping Report, Rubrics Statistic Summary Report, Standards and Outcomes Alignment Report, Standards/Outcomes Report and the Student Profile Report.

C1 Assessment Report

The C1 Assessment Report can be used by both faculty and administrators to demonstrate student growth and levels of achievement on assessments. This report can be tailored to show the data collected for standards, courses, assessments, and assignments. Faculty and Administrators can choose to filter the information by dates, assessment type, and scoring type.

Faculty and administrators can cross reference assessment data with data collected in specific fields from LiveText forms (if the form was launched from their account). This is often used to separate assessment data by program or other data points. The requirement is that the individual or individuals being reported on must take the form and be assessed with the rubric. Any individuals who did not take the form will be excluded from the assessment report the moment the form is added. The form dataset must reside in the Admin Account and the dataset cannot be archived. Additionally, the student profile filters can be used to disaggregate assessment data for further analysis.

To create a C1 Assessment Report:

  1. Click the Tools tab located at the top of the screen.

  2. Go to the Reports tab, click the New button and select Assessment Report under the category --C1 only--

    The Tools page showing the Reports tab selected. The Reports section displaying a New dropdown with report options, along with Export Assessments, Export Rubric Standards, Export Course Assessments, and Delete buttons. A table below showing report rows with Edit and Rename links in the Actions column.

  3. Enter a Title

  4. Enter a Description (Optional)

    The Assessment Report section showing the General Information section with fields for Title and Description, both empty.

  5. Select your Course Filters: Terms
  6. Select your Course Filters: Colleges
  7. Select your Course Filters: Departments
  8. Select your Course Filters: Location(s)
  9. Select your Course Filters: Course Sections
  10. Select your Assessment Filters: Assessment Rubric, Assessors

  11. Select the Assessment Type from the dropdown menu and select Formative or Summative. When no type is selected, the report will contain both Formative and Summative data

    The Assessment Filters section showing fields for Assessment Rubric, Assessor, Assessment Type, and Scoring Type. Additional options displaying Inter‑Rater Summary set to Yes and a Date Range with From and To fields.

  12. Select Scoring Type from the dropdown menu and select Draft or Final. If no type is selected, the report will contain both Draft (Request Resubmission) and Final (Submit Assessment) data
  13. Select Inter-Rater Summary (optional and only available in an administrator account)
  14. Select Date Range (optional)
  15. Add Form-Based Filters (optional)

  16. Click View Report and the assessment data will be displayed visually in two formats: A table which represents the selected rubric section of an assessment document with its performance levels and elements. The numerical values displayed under the levels represent the total number of assessments scored at that level. A color-coded graph displays those same assessment totals by percentage

    The rubric table with elements listed in the first column and counts for Emerging, Developing, and Mastering levels in the next columns. Additional columns showing Mean, Mode, and Stdev values for each element. A bar chart below displaying each element with colored bars representing performance levels, each bar labeled with counts and percentages such as 1 (33%) and 2 (66%).

  17. To save the report for generating again at a later time, click the Save As button
  18. Click Export to CSV button to export data in a CSV

Curriculum Mapping Report

This report is designed to map the location of standards and outcomes to rubrics and other content with which they are associated. Administrators can generate a report that shows if standards and outcomes are mapped to a course, assignment, and/or rubric. This report is useful for all programs or institutions that do not have a curriculum map defined or fully developed. Results from this report can drive change and program improvement by displaying where standards and outcomes have been successfully aligned or not aligned at all.

Note: Standards and outcomes must be aligned to one of these areas in order for this report to be generated.

To create a Curriculum Mapping Report:

  1. Click the Tools tab located at the top of the screen
  2. Go to the Reports tab, click the New button and select Curriculum Mapping Report under the category --C1 only--
  3. Enter a Title
  4. Enter a Description (optional)
  5. Select Criteria: Standard/Outcome Sets
  6. Select Criteria: Terms
  7. Select Criteria: Departments
  8. Select Criteria: Colleges

  9. Click View Report button located in the lower right corner of the report area. Using the legend, this report will display where a specific standard is associated; either at the rubric, assignment, or course level, or all three levels

    The table showing rows with columns for Standard Set, Standard, Course, and Section. The Alignment column displaying icons for Course, Assignment, and Rubric on each row.

  10. To save the report for generating again at a later time, click the Save As button
  11. Click the Export to CSV to export data in a common file format, CSV. This file can be imported into most data analysis tools

Rubric Statistics Summary Report

This report allows the administrator to generate a report that compares performances by gender and ethnicity. This specific report will provide entities with the information they need to make improvements and changes as they grow and the student population increases.

To create a Rubric Statistics Summary Report:

  1. Click the Tools tab located at the top of the screen
  2. Go to the Reports tab, click the New button and select Rubric Statistics Summary Report under the category --C1 only--
  3. Enter a Title
  4. Enter a Description (optional)
  5. Select the Institution
  6. Click the Student Gender dropdown menu and select Male or Female. If no gender is selected, the report will contain both Male and Female data
  7. Select one or more ethnic groups from the Student Ethnicity list box. If no Ethnic Groups are selected, the report will contain all Ethnicity data

  8. Enter the Date Range to specify the assessment period

    The Criteria section showing dropdowns for Institution and Student Gender, with Institution set to C1 University and Student Gender set to Female. The Student Ethnicity field displaying a multi‑select list of ethnicity options. The Date Range field showing From and To inputs.

  9. Click the View Report button located in the lower right corner of the report area.
  10. To save the report for generating again at a later time, click the Save As button

  11. Click Export to CSV button to export data in the common file format, CSV. This file can be imported into most data analysis tools

    The table showing rows for different rubrics with columns for Rubric ID, Rubric Title, Count, Mean, Min, Max, and Std Dev. The top right showing the Export to CSV button highlighted.

Standards & Outcomes Alignment Report

This report shows how standards have been aligned to courses, assessments, and assignments. Administrators can use this data as a reference to determine if they have successfully aligned standards and outcomes

To create a Standards & Outcomes Alignment Report:

  1. Click the Tools tab located at the top of the screen
  2. Go to the Reports tab, click the New button and select Standards & Outcomes Alignment Report under the category --C1 only--
  3. Enter a Title
  4. Enter Description (optional)
  5. Select the Institution
  6. Select a Standard Set to be included in the report
  7. To generate a report that queries standards associated with Courses, Assignments, or Assessment Rubrics, select the type of association from the Associated with dropdown menu. If no choice is selected, the report will contain all associations

  8. Click the View Report button located in the lower right corner of the report area

    The section showing fields for Standard Set and Associated with, followed by action buttons on the right. The View Report button highlighted next to Save As.

  9. To save the report for generating again at a later time, click the Save As button

  10. Click the Export to CSV button to export data in the common file format, CSV. This file can be imported into most data analysis tools

    The table showing rows with columns for Standard Set, Standard, Associated with, Description, Use, and Course Info/Assessment Title. The Export to CSV button highlighted at the top and bottom of the table.

Student Progress Report

This report allows administrators to track a student's completion of key/signature assessments that are aligned to required standard sets. Administrators can use this report to view a student’s progress and to see if the student has reached key points/ benchmarks within the program. Instructor’s comments are provided to show the student’s growth when assessed multiple times, as well as the name of the assessment.

Note: Rubrics MUST be aligned to standards in order to generate data for this report.

To create a Student Progress Report:

  1. Click the Tools tab located at the top of the screen
  2. Go to the Reports tab, click the New button and select Student Progress Report under the category --C1 only--
  3. Enter a Title
  4. Enter Description (optional)
  5. Select the Institution
  6. Type in the first few letters of the student name or username. The auto complete system predicts the rest of the entry and generates a list of 15 possible completions from which to choose
  7. Select a Standard Set from the dropdown menu that will demonstrate how the student met each standard

  8. Click the View Report button located in the lower right-hand corner of the report area

    The section showing fields for Student and Standard Set, with Student set to Student, Olivia (username – oliviastudent) and Standard Set set to CACREP‑2016. The View Report button highlighted next to Save As.

  9. View Options: A student’s name can be hidden from view, and the user can toggle between viewing Points attained or Performance Level attained. Hiding the student name is useful when sharing reports with faculty or external reviewers where the student identity is not to be disclosed
  10. To save the report for generating again at a later time, click the Save As button
  11. Click the Export to CSV button to export data in a common file format, CSV. This file can be imported into most data analysis tools

Standard/Outcome Report

This an assessment report which maps the performance levels from disparate rubrics to common performance levels, then aggregates the data for all rubrics across the selected standards/outcomes. The assessment data comes from the assessment of course assignments and document review submissions. The report allows assessed data to be filtered by many criteria including rubric, assessment type (summative or formative), assessment status (draft or final), term, gender, etc.

To create a Standard/Outcome Report:

  1. Click the Tools tab located at the top of the screen
  2. Go to the Reports tab, click the New button and select Standard/Outcome Report under the category --C1 only--
  3. Enter a Title
  4. Add Description (optional)
  5. Check the box to make this report available in the ExhibitCenter (Optional)

  6. All the standard sets which have been aligned to rubrics in the selected domain are available in the Standard Set dropdown list. Select the set which contains the standards you wish to include. Only one set can be selected on this screen

    The Standard/Outcome Report page showing the General Information section with empty fields for Title and Description and Institution displayed below. The section showing a Standard Set dropdown highlighted.

  7. You can either Add the Entire Set or Refine Standards and search for specific standards within a set

    The Standard Set: CACREP‑2016 section showing action buttons for Cancel, Add the Entire Set, and Refine Standards.

  8. Option 1: Click the button to Add the Entire Set

  9. Click Finish Defining Standards. Additional Standard Sets can be added at this time by selecting from the Standard Set dropdown menu

    The Standard/Outcome Report page showing the Standard Set section with CACREP‑2016 selected. The page displaying a table listing added standards and a note about adding more. The Finish Defining Standards button highlighted at the top and bottom of the standards list.

  10. The total number of scored rubric documents aligned to defined standards will be displayed on this screen

  11. Option 2: Click the button from the Standard/Outcome home screen to Refine Standard(s)

    The Refine Standards for set: CACREP‑2016 section showing a keyword search field labeled Search Standard by Keywords, with a note explaining how to search. The section displaying Cancel and Search buttons.

  12. Search Standard by Keyword to select specific standard(s) within a set

  13. Click the Search button

    The Refine Standards for set: CACREP‑2016 section showing the keyword search field with Student entered. The results area displaying Results found: 1 with an Add Selected Standards button above the table. The table showing one row with Standard 1.A and its description. The section displaying Cancel and Search buttons.

  14. Select the specific standards that you would like to add to the report by checking the box next to the standard and click the Add Selected Standard(s) button

  15. Click Finish Defining Standards

    The Standard/Outcome Report page showing the Standard Set section with CACREP‑2016 selected and Total Standards Added: 1. The table displaying Standard 1.A with its description. The Finish Defining Standards button highlighted at the top and bottom of the section.

    The total number of scored rubric documents aligned to defined standards will be displayed on this screen.

  16. Select the rubrics to be included by clicking the checkbox in the left column

    The Assessment Rubrics section showing a table listing rubric records with columns for Standard, Assessment Document, Rubric Title, Creator, Rubric Source, and Rubric Levels. Two rows selected at the left. The section showing Add Optional Filters on the right and the Finish Selecting Rubrics button.

  17. If desired, optional reporting criteria can be defined by clicking the Add Optional Filters button
  18. Add the filter in the column on the left, and choose the data you wish to include in the report in the column on the right. Multiple filters can be applied at the same time

  19. Click Finish

    The Optional Filters modal showing Step 1 with a filter list under Institution Criteria, including options like Academic Class, Academic Program, Assessment Type, Concentration, Degree Program, Department, and College. Step 2 displaying an empty Criteria Options panel. The modal showing Cancel and Finish buttons on the right.

  20. Click Finish Selecting Rubrics when all the criteria and filters have been selected

    The Assessment Rubrics section showing the summary tiles at the top with Optional Filters set to 2 and an Edit button below it. The table below displaying rubric records with selection checkboxes. The section on the right showing Add Optional Filters, and the Finish Selecting Rubrics button highlighted.

  21. Assessment data can be normalized from rubrics with different performance levels by selecting a value from the Standard Levels dropdown menu
  22. Select the number of performance levels that will be common for all normalized rubrics
  23. Type the names of the normalized performance levels into the text boxes

  24. Click Apply Standard Levels to complete the mapping on the next page

    The Create Standard Levels section showing a Standard Levels dropdown set to 4, with four empty label fields displayed in a row. The section showing the Apply Standard Levels button on the right.

  25. Map each rubric level into the normalized performance level under the Standardized Level column. All levels must be mapped for selected rubrics or the report cannot be run

    The Mapping Rubric Levels to Standard Levels section showing a table with rows for two Critical Thinking Rubric records. Columns displayed for Assessment Document, Rubric Title, Creator, Rubric Source, Rubric Level, and Standardized Level. The Standardized Level dropdown open for the Emerging row, showing options Unacceptable, Acceptable, Proficient, and Accomplished. The View Report button appearing at the top right.

  26. Click the View Report button

Standard/Outcome Report Results

The Standard/Outcome Report page showing the results table with columns for Standard, Rubric Count, Total Assessments Count, Total Students Count, and level columns for Unacceptable, Acceptable, Proficient, and Accomplished. The Total Assessments Count and Total Students Count columns highlighted.

Rubric Count

The number of unique rubrics in which the listed standard is included.

Note: The same standard may occur multiple times in a single rubric, but the rubric will only be counted once.

Total Assessment Count

The total number of assessments that are aligned to the listed standard.

Note: This count may include multiple assessments of the same student.

Total Student Count

The number of unique students that have been assessed with a rubric aligned to the listed standard.

Note: This count includes students that were assessed at least once. Students who have been assessed more than once for a listed standard are still only counted once.

Performance Levels

The first number represents the level assessment count, which is the number of assessments scored at each level. The second number, in parenthesis, represents the level student count, which is the number of unique students assessed at each performance level.

Note: Students assessed at more than one performance level will be counted in the level student count for EACH level at which they were scored.

Export to CSV

The table is exported in CSV format with each number in a separate cell. The output can be opened in a spreadsheet application like Excel to allow for graphing, sharing, or statistical analysis.

Was this article helpful?
0 out of 2 found this helpful

Articles in this section

How to Contact Support
There are many ways to reach out! Click the icon above for our support options.
Watermark Academy
Click the icon above to access the Watermark Academy for consultation, training, and implementation companion courses.
Customer Community
Can’t find the answer? Ask fellow users how they’re making the most of Watermark in our Community!