Background
The Kuali Research HR API is designed for integration with your local HR system. The API receives XML and loads the appropriate Kuali Research tables to prepare your instance to be used by your research faculty and staff. Subsequent management of HR data can be done through the application or via the API. More information on our APIs can be found in the API Overview article.
Many institutions choose to automate the periodic loading of data via a job scheduled within their on-premise IT infrastructure. This is the most efficient way to ensure that your HR data is up to date and loads without error.
The HR feed will populate the Person and Person Extended Attributes tables in Sponsored Programs and then push the user records to Kuali Users to manage authentication. It's a two stage process for the records to get populated from the HR XML feed sent to Kuali Research. If you only have Compliance modules you won't need to populate Person and Person Extended Attributes tables and only a Kuali User is require. The flow of the HR feed process is illustrated below:
Below outlines the HR API configuration information and how to get that set up for your institution. Information on creating the API key/account can be found in the Setting Up and Maintaining an API Key article.
Usage
Security
Kuali Research APIs used for integration with local systems are secured industry standard JWT API tokens, which are unique to each hosted instance. More information on API key management can be found in the Setting Up and Maintaining an API Key article.
Data Elements
Required data fields will be different depending on which Kuali Research modules and functionality will be implemented at your institution. Some fields are not strictly required by the system but are extremely useful. HR data categories are: Required, Recommended, Grants.gov S2S, and Optional. Some data elements must be entered in pairs. For example, Phone Number is an optional field, but if you choose to leave Phone Number blank, then Phone Type Code should also be blank. A full list of fields for each category is included in Appendix A.
Formatting your HR data
The HR API requires data to be formatted and uploaded as XML. The XML schema defines the format and valid data values for fields that refer to other Kuali Research fields. For convenience we provide a Ruby Gem to convert from CSV to XML.
XML
A sample of the required XML for a single individual is included in Appendix B. Certain data elements can have multiple entries per individual listed in the data feed. For example, Faculty members may have multiple appointments with different salary amounts (academic year and summer appointments are a common reason for this). To accommodate this scenario, multiple person appointment records can be added to a single person record in the XML. Other elements that can have multiple entries include Name, Address, Phone, and Email. Refer to the XML schema for a complete description of the data.
CSV
A sample CSV file is included in Appendix C. The CSV file can be used to add multiple appointments via adding two lines within the file. Other elements that can accommodate multiple entries in the XML feed are not supported in CSV. If you have a need to provide additional Names, Addresses, Phone Numbers or Emails, then your HR data must be loaded via XML. The CSV file must have proper headers defined for each row included in the feed. The CSV column headings for each data element is included in Appendix A.
Submitting your HR data
Preparing your Environment
Make sure you have a Ruby runtime on the machine where the conversion will occur. Instructions on installing Ruby can be found here: https://www.ruby-lang.org/en/documentation/installation/.
Please note - the HR API integration is confirmed to run on Ruby versions 2.1.2 --> 2.1.5 or 2.4.1
Once you have Ruby installed, from your command line run:
> gem install kuali_toolbox
Server response:
Fetching: kuali_toolbox-0.42.gem (100%)
Successfully installed kuali_toolbox-0.42
Parsing documentation for kuali_toolbox-0.42
Installing ri documentation for kuali_toolbox-0.42
1 gem installed
In addition to the Kuali Toolbox gem, this process will install two executables:
transform_CSV_to_HR_XML and validate_HR_XML.
Note: More gems may be installed if you are missing required dependencies.
Note: Depending on your platform and rights, you may need to run the gem install command as a super user. If you’re not sure whether or not you need to do this, ask a system administrator at your institution.
Converting from CSV to XML
If you choose to start with a CSV file and convert it to XML, you can do so very easily.
Once you have the Kuali Toolbox gem installed, convert it to XML with the following command:
> transform_CSV_to_HR_XML .csv
Server Response:
XML file written to ./.xml
Congratulations! The XML file passes XSD schema validation! w00t!
If there are warnings or errors during the conversion process, you’ll be notified in the console. An example follows:
WARN: Line 149: Data will be truncated: DIRECTORY_TITLE.length > 50: 'Professor of Health, Human Services & Public Policy'-->'Professor of Health, Human Services & Public Polic'
WARNING:Duplicate employment info for single affiliation. Skipping extra employment info on 721
In this case, you can see that the process is only giving you warnings. If there are errors that prevent the processing of the data or creation of the XML file, you will be notified of the error and where in the CSV file the error occurred. Errors will prevent the XML file from being created. Warnings will proceed with XML file creation.
Uploading to your Kuali Research instance
There are multiple ways to call REST service APIs. Depending on your IT infrastructure you may have preferred tools or you may have a cron or batch job to automate this process. For simplicity, the example below uses the *nix cURL command to submit an XML file to a Kuali Research instance and assumes a manual load of data into the API. Submit a service request to the Kuali Research team to get the specific username, password, and server address for your instances. The server responds in JSON (JavaScript Object Notation).
In this example, the parameters used in the command are:
API Key: 123.456.789
Server: univ-sbx.kuali.co
Path on Server: /res/hr-import/hrimport/import/
File Name: sample-file.xml
> curl -H "Authorization: Bearer 123.456.789" --form file=@sample-file.xml https://univ-sbx.kuali.co/res/hr-import/hrimport/import
Server Response:
{"importId":"11111111-2222-3333-4444-555555555555","status":"PROCESSING","message":"Import is processing","startTime":"1442368887266","recordTotal":"1337","processedRecords":"0","errorCount":"0"}
The API has accepted your file, assigned it an importId, and is responding with the status of PROCESSING and the number of records present in the file.
Restrictions
Be aware of the below restrictions on the HR feed:
- The system only allows one job to run at a time so if you attempt to send another one while one is already running it will fail immediately.
- The payload requires more than one user to be included in the feed and if it contains no users (blank) or only one user it will fail immediately. This is to help prevent sending blank or incomplete files that could inactivate users inappropriately.
Monitoring the progress of your HR load
Within the XML you can set an email address to receive notification of completion of the HR feed or if any errors arise: statusEmailRecipient="no-reply@kuali.co"
. If you are using the CVS to XML then they can use a command line arg to set the email address --email no-reply@kuali.co
You can also use the importId returned with the initial upload to check on the status of your load. Responses are in JSON strings.
The same username, password and URL are used to check the status, only with the importId appended to the end. You do not need to send the --form or @file elements when checking on the status.
> curl -H "Authorization: Bearer 123.456.789" https://univ-sbx.kuali.co/res/hr-import/hrimport/import/
Server Response:
{"importId":"11111111-2222-3333-4444-555555555555","status":"PROCESSING","message":"Import is processing","startTime":"1442351305258","recordTotal":"1337","processedRecords":"189","errorCount":"0"}
If the API runs into issues loading any of the records in your feed it will continue processing your file. When you check the status of the run it will include an error count and the reason for the error. For example:
{"importId":"11111111-2222-3333-4444-555555555555","status":"COMPLETE","message":"Import completed normally","startTime":"1442351305258","endTime":"1442352328202","recordTotal":"1337","processedRecords":"1337","errorCount":"1","errors":[{"recordNumber":"987","principalName":000222444","exception":{"type":"IllegalArgumentException","message":"Unit number 12234 does not exist"}}]}
Which parses to:
{
"endTime": "1442352328202",
"errorCount": "1",
"errors": [
{
"exception": {
"message": "Unit number 12234 does not exist",
"type": "IllegalArgumentException"
},
"principalName": "000222444",
"recordNumber": "987"
}
],
"importId": "11111111-2222-3333-4444-555555555555",
"message": "Import completed normally",
"processedRecords": "1337",
"recordTotal": "1337",
"startTime": "1442351305258",
"status": "COMPLETE"
}
The job is letting you know that it encountered an error because person record 987 referenced a nonexistent Unit.
Another common issue is collisions between manually entered people and people added via the HR feed. The API will report that it was unable to load a record in that case.
If errors are encountered during the HR feed, the status will be set to "Abnormal_Termination". This means some records were not processed due to errors, but the feed still finished. However, if a system interruption, such as a server issue, no status email will be sent. The lack of an email is the main sign that the HR feed did not complete.
Appendix A - Data Elements
HR API Data Elements (CSV) - see tab 1 of attachment included at the end of the article.
Note: DO NOT submit a record with a principalId of 1. There is an out of the box system user that utilizes this principalId and if this user is overwritten by the HR feed it will break some document processing functionality.
Appendix B - XML
Sample XML for a single record:
<?xml version="1.0" encoding="UTF-8"?>
<hrmanifest xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://github.com/KualiCo/ce-tech-docs/tree/master/v2_0 https://raw.github.com/rSmart/ce-tech-docs/master/v2_0/hrmanifest.xsd" xmlns="https://github.com/rSmart/ce-tech-docs/tree/master/v2_0" schemaVersion="2.0" statusEmailRecipient="no-reply@kuali.co" reportDate="2015-09-15T14:03:30-04:00" recordCount="5519">
<records>
<record principalId="000222444" principalName="superuser">
<affiliations>
<affiliation affiliationType="STAFF" campus="UNIV" default="true" active="true">
<employment employeeStatus="A" employeeType="O" baseSalaryAmount="10000.0" primaryDepartment="000001" employeeId="000222444" primaryEmployment="true"/>
</affiliation>
</affiliations>
<addresses>
<address addressTypeCode="WRK" addressLine1="1122 Boogie Woogie Ave" city="Anytown" stateOrProvince="UT" postalCode="12345" country="US" default="true"/>
</addresses>
<names>
<name nameCode="PRM" firstName="Super" lastName="User" default="true" active="true"/>
</names>
<emails>
<email emailType="WRK" emailAddress="kc-support@kuali.co" default="true" active="true"/>
</emails>
<kcExtendedAttributes primaryTitle="Support" citizenshipType="1"/>
<appointments>
<appointment unitNumber="000001" appointmentType="6" jobCode="AA000" salary="10000.0" jobTitle="Support"/>
</appointments>
</record>
</records>
</hrmanifest>
Appendix C - CSV
Sample for a single record (CSV) - see tab 2 of attachment included at the end of the article.
Document Revision 1.2
Comments
0 comments
Article is closed for comments.