Cloud Connect Requirements

Overview

Data is imported to Student Success & Engagement (SS&E) via CSV files which are SFTPed by the institution to the SS&E AWS S3 data store. The Cloud Connect (Accelerate) process is hosted in the Watermark AWS environment, where it transforms the data to match the import requirements (SIS Import Definitions)

This guide provides information on the Cloud Connect requirements for an institution. The details outlined in this guide are subject to change as Watermark processes and requirements change over time.

Your Implementation or Consulting team will provide you with specific connection details when you set up Cloud Connect.

Cloud Connect requires an institution to extract CSV files from the SIS and SFTP them to Watermark to facilitate the data transfer. This should be an automated process, not a manual one. For schedule recommendations, please take a look at CSV Extract Frequency. 

CSV Upload Requirements

File Location

Each unique schema file will need to be placed in a separate folder. If you are following the SS&E SIS Import Definitions then at a minimum, you would have a folder for each of the datafeeds you are implementing. 

• The data feed folders should be camelCased. 
  For example the academic calendar folder should be called “academicCalendar”
• If you have a single file for a datafeed then you would place only a single file in the feed folder.
  For example the course section feed file would be uploaded to courseSection/course_section.csv
• Please create a folder for the feed only when you have a file to send. ( Avoid creation empty folders for the feeds. )
• If you have multiple files for a datafeed AND they share the same schema then they can be placed in the same folder.
  For example if your person feed is going to be made up of 3 files (students, staff, and faculty) then you could place all the files into the same folder as long as the files all had the same columns in them:
  • person/person_student.csv
  • person/person_staff.csv
  • person/person_faculty.csv
• If you have multiple files for a datafeed and they have different schemas then they must be placed in different folders.
  For example if your person feed is going to be made up of 3 files (students, staff, and faculty) with different columns, they would be placed in separate folders:
  • personStudent/person_student.csv
  • personStaff/person_staff.csv
  • personFaculty/person_faculty.csv
• If you have custom files then you should work with your implementation team to agree on a folder name.

File Names

• CSV files should be named using snake_case.
  For example, the academic calendar csv file should be named academic_calendar.csv
• If you have custom files then you should work with your implementation team to agree on a custom file name.

File Format

• CSV files must have column headers
  • These headers should match the headers specified in the SIS Import Definitions 
  • If you have custom fields then you should work with your implementation team to agree on a custom header name.
• CSV files must follow the CSV standard. Some important considerations of the standard
  • It is recommended that all fields be enclosed in double quotes.
  • Strings should be enclosed in double quotes.
  • Strings that contain a double quote, comma or line break MUST be enclosed in double quotes
  • Double quotes within a string MUST be escaped with a second double quote.
    Example: The string `“This is a test” is a fun phrase`
    Would look like this: “””This is a test”” is a fun phrase”