District and School administrators can use CSV files to load their roster data into Passport. Upload of CSV files is an alternative to API-based data synchronization. One should use CSV files in the absence of support for API based data synchronization for the given SIS or LMS system. CSV files can be upload into Passport either through Admin UI or via SFTP.
Passport supports several types of CSV data schemas. Each CSV data schema requires a different set of files and attributes in the files and has an associated data validation rules.
Passport supports the following CSV schemas:
- Passport Basic Identity Data - a basic set of data mostly appropriate for provision of Passport user accounts
- Passport Roster Data - supports most complete set of data including information about Students, Teachers, Administrators Schools, Sections and Enrollments
- Passport SSO Credentials - allows to pre-populate application credentials to establish “Password Vault” based SSO connectivity for existing Passport accounts
Notice: not all supported CSV data schemas might be available for a given organization. Contact your service provider to request configuration changes.
One can download CSV templates for each CSV schema:
- Go “Users” > “CSV Upload” to open the “CSV Upload” screen
- Choose preferred data model in “Data model” drop-down box
- Click the “Download Templates” link and save zip archive containing templates to your local drive.
Upload of data can be Incremental or Full . With Full mode systems expects that upload contains a complete set of data. Thus, Passport can determine what data are new, updated or missing compared to previous upload and, thus, subject for deletion. With Incremental mode Passport expects only new and updated data in the uploaded CSV files. By default all institutional accounts are configured for Full upload. Contact your service provider to request configuration changes.
CSV Upload Tool
Navigation: Users > CSV Upload
CSV Upload Tool allows to manually upload CSV files and validate data. CSV upload process consists of several steps. In each step you can stop upload process by clicking “Reset”.
1. Choose CSV files to upload using one of the options:
- Click “Add files” button to open a Open File dialog window, where one can select one or more CSV files to upload to Passport.
- Click the “Add File” icon (+) next to the file type you want to upload and select a single file.
2. Upload and Validation .
System uploads files to a secure storage and starts the data validation process.
If data validation is successful for a given CSV file, Passport displays “Validated and Uploaded”.
If data validation errors are detected, “Validation failed” message gets displayed for the uploaded file and an arrow appears under the filename. Clicking this arrow opens an expanded view with a detailed breakdown of the errors. Click the Reset button to reset the upload, correct the error(s) in the CSV file, and then attempt the upload again. For certain types of errors Passport allows to proceed to the next step of data upload process. See "Validation error levels" section and "CSV Files schema validation" section for further details.
3. Verify Changes . This step allows to verify the number of data objects that will be added, removed or updated as a result of CSV file upload. Clicking the down arrow expands the list to show a detailed view of each change.
4. Commit to Production . Finalize the changes by clicking “Commit” button. Only after that changes are applied to system data. Passport utilizes event-driven architecture - the data get propagated across different components of Passport asynchronously. It may take up to a few minutes until all system components get updated.
Validation error levels
Every type of errors has the assigned Error Level attribute. Error Level defines how system will treat the data in CSV files. Passport defines the following Error Levels:
- Error (red colored) - any error of with this level invalidates CSV upload session. Errors should be corrected to proceed with upload.
- Warn (orange colored) - system will allow to proceed with upload of data by removing corresponding records from CSV files before further processing.
- Info (blue colored) - system will save data as provided in CSV file.
By default Passport applies the following configuration of error types:
- Error level for "Required attribute", "Invalid Format", "Uniqueness violation" error types;
- Warn level for "Reference not found" error type.
Notice: Error level mapping can be adjusted. Contact your service provider to request configuration changes.
Validation errors report is accessible from Version History view.
The “ Version History ” table lists the history of uploads (including uploads via SFTP) and contains links to download CSV files
“Version History ” contains the following information for each CSV upload transaction:
- Version ID: system generate version number for the upload transaction.
- Commit Date and Time: date and time of the upload.
- Committed by: Email of the user account who uploaded the file.
- Status: status of the transaction. This will be one of the following:
- Pending propagation: upload transaction is in progress.
- Propagation failed: An error occurred. Contact Technical Support for assistance.
- Successfully in production: for successful upload transaction.
- Validation Errors: validation errors detected.
- Download: clicking this link allows to download the original .CSV file(s) in compressed (ZIP) format and save it to the specified location on your computer or local network.
To view validation errors that happened during CSV upload find respective upload session in Version History table and click on "Validation Errors" link in Status column. This will start download of report with data validation errors.
CSV files schema validation
Validation of input data include the following types of check:
- Required value (mandatory). If required value is missing then validation of data fails and further steps of CSV upload are unavailable.
- Duplicates. If validation process identifies multiple records with the same values of key fields (for example, Students.csv contains two records with the same Student_id and School_id) then validation of data fails and further steps of CSV upload are unavailable. A user account in a district wide upload can be associated with more than one school.
- Cross file entity references allows to detect issues when a data object references non-existing entity (for example, Student_id in a Enrollment object is not defined in the list of Students). Passport applies permissive control to Cross-Entity validation - Passport generates report about issues with cross-check validation and filters respective records out from the upload while the rest of the data get persisted in the system.
Following are the rules that govern cross-reference errors handling:
- student record will be filtered out if students.school_id is not found in schools.csv
- teacher record will be filtered out if teachers.school_id is not found in schools.csv
- admin record will be filtered out if admins.school_id is not found in schools.csv
- section record will be filtered out if sections.school_id is not found in schools.csv
- enrollment record will be filtered out if enrollments.school_id is not found in schools.csv
- enrollment record will be filtered out if enrollments.section_id is not found in sections.csv
- enrollment record will be filtered out if enrollments.student_id is not found in students.csv
Refer to Passport CSV schema for information about supported CSV Schemas.