Import Process Overview
Data is imported to SS&E via CSV files which are SFTPed by the institution to the SS&E AWS S3 data store. The Cloud Connect (Accelerate) process is hosted in the Watermark AWS environment where it transforms the data to match the import requirements.
The SIS data is then imported from AWS S3 into the SS&E database via the https://yourInstitution.avisoapp.com/aviso/springBatchJob/show/sisImportBatchJob so that the imported data becomes available on the UI and visible from the https://yourInstitution.avisoapp.com/aviso/datafeed/list.
Note: Replace yourInstitution in the above link with your SSE/Aviso domain as displayed in a working URL. For more details, click Support Articles include Broken Links with 'yourInstitution'.
Legacy methods for importing data (Connect 1 and Connect 2) are discussed at the end of this article.
Data Flow Diagram
Note: see downloadable image at end of article.
Additional Information
Schedules
The Connect scheduled task is scheduled to run four times a day, so that SIS data can import up to four times a day, depending on how long it takes each import job to complete.
- The CSV extract schedule is entirely up to the institution and should align with the Connect/Accelerate schedule where possible. For example, 1am, 7am, 1pm, 7pm. See SS&E CSV Extract Frequency for suggestions on custom schedules if needed.
- The Cloud Connect job runs at 2am, 8am, 2pm, 8pm by default.
- The SIS Import Job runs at 4:30pm, 10:30pm, 4:30pm, 10:30pm. This schedule may not be changed.
Accelerate XML Files
If anything needs to be changed, this is entered into the Child XML which is run first.
If code is not found in the Child XML, then the Parent XML (default Accelerate code) is used to process the data feed.
Import Logging
Import Errors can be found in SS&E Administration SIS Integration Import Errors, and Import logs can be found in SS&E Administration Advanced Batch Processing
Advanced Import Caching
SSE stores a hash key in our cloud to detect differences in incoming data. If there are no differences in a record since the last time we encountered it, it will not reimport
Clearing cache(s) via SSE administration will cause a full data import to take place. It is not recommended to clear all caches at once.
Legacy Connect Methods
Before Watermark installed cloud-based infrastructure for ingesting, transforming, and importing data via the AWS cloud, there were two methods for implementing Student Success & Engagement SIS Data Integration on the institution's Connect server. Customers implemented before fall 2025 will be using one of the legacy methods and may continue to do so indefinitely.
- Connect Method 1 uses CSV files (often exported from the SIS via Informer or another custom export method) as the data source. The Connect2 (Accelerate) process then SFTPs data from the institution's Connect server to the SS&E AWS S3 data store, optionally transforming the data to match the import requirements.
- Connect Method 2 is a direct database connection using SQL Views or Queries. The Connect2 (Accelerate) process then SFTPs data from the institution's Connect server to the SS&E AWS S3 data store, optionally transforming the data to match the import requirements.
Data Flow Diagrams
Connect Method 1: CSV Files
Note: see downloadable image at end of article.
Connect Method 2: Direct Database Connection
Note: see downloadable image at end of article.
Schedules
The Connect2 scheduled task is scheduled to run four times a day, so that SIS data can import up to four times a day, depending on how long it takes each import job to complete.
- The CSV extract schedule is entirely up to the institution and should align with the Connect/Accelerate schedule where possible. For example, 1am, 7am, 1pm, 7pm. See SS&E CSV Extract Frequency for suggestions on custom schedules if needed.
- The Connect2 job runs at 2am, 8am, 2pm, 8pm by default. The schedule may be customized by the institution for those running Connect2 locally on their Connect Server.
Connect2 Job Logging
See more information about External Logging in Connect Job - SS&E External Logging.