Splitting Reference Data

When uploading reference data, it’s important to keep in mind that you are limited to a 1mb file, or a 15mb file if you’re separating the data by user groups (See here for more information). However, if you have a dataset that grows beyond this size, and you find you are unable to reduce it, there is a way to work around these file size limits.

Creating the second file

Since you already have a single reference data file, the first step is to create the second copy where you can continue to add new records and upload it to GoCanvas. Ensure that you use the exact same column headers as in the first file. If the first file exceeds the size limit, copy enough records from the first file to the second to populate it with some data.

Alternatively, you can remove columns from one file and move them to another. You will need a common column to be used as the key column in both files in order for the files to work correctly together. 

Editing your forms

The real challenge with this solution begins when editing your forms to use both files. Because a single field cannot use more than one reference data file, this means you will need to create two fields to use those files. Of course, this can generate a lot of clutter. To address this, we will need to first identify a column in the dataset we can use to define a range of the data and determine which fields are going to be used.

Example: let’s say your user has to input the product ID to get reference data to populate in your form. In the first reference data file, let’s say the product ID ranges from 1 to 1000. Therefore, if the ID is greater than 1000, we need to look at the second file.

This means that for the other fields that look at the product ID, we need two copies that both reference the product ID field, but the first looks at file 1, and the second looks at file 2. Using conditions (Learn how to set up conditions), we can set it to show the second field if the value of Product ID is greater than 1000, or alternatively, it shows the first one if it’s less than or equal to 1000. Repeat this with all fields linked to your initial reference data file.

By doing the above, you have effectively created a form that can use a dataset range that works with multiple reference data files. While it’s always preferable to try to reduce the overall file size, this solution can be used in extreme situations where the size limit isn’t sufficient to contain your whole dataset.