Strategies for Managing Large Reference Data Sets

Effectively managing large reference data sets is crucial for GoCanvas users who rely on dynamic data for streamlined business processes. With reference data powering your various fields and form logic, handling volume efficiently ensures smooth mobile experiences, accurate data collection, and faster workflow automation. This article explores proven strategies tailored for managing large data sets within the GoCanvas platform, helping your business maintain efficiency and accuracy.

Understanding Reference Data in GoCanvas

Reference data in GoCanvas serves as the backbone for dynamic form fields, ensuring accurate and up-to-date information follows your processes. Large data sets may include thousands of product SKUs, client lists, or equipment inventories that enable lookups, minimize input errors, and drive business logic within your digital forms.

Best Practices for Uploading and Structuring Large Data Sets

• Use CSV files for batch imports—ensure they’re properly formatted and free of duplicates.
• Break data into logical categories (e.g., by region, brand, or product line) for faster loading and easier maintenance.
• Leverage unique IDs and relationships to link reference data tables, minimizing redundancy within the platform.

Split your Reference Data Files

Splitting reference data in GoCanvas is necessary when your dataset exceeds the allowed file size limit, allowing large datasets to work across multiple files while maintaining proper functionality and reducing performance bottlenecks.

Key Points to Use

• Reference data file limits: 1MB per file, or 15MB if split by user groups.
  • See this article for more information regarding splitting reference data by User Group
• If data exceeds limits, create a second reference file with identical column headers for added records.
• Splitting columns between files is possible; ensure a common column (key column) is present in both files so they can function together.
• When editing forms, each field can only reference one data file, so duplicate fields may be needed to use both files without losing functionality.
• Use conditional logic to display one field or the other depending on data value (e.g., show second field when Product ID > 1000).
• Proper formatting of CSV/Google Sheets is essential for reference data files to work; in-depth instructions are available on the GoCanvas help site.
• Reducing overall file size is preferable, but splitting data files is a workaround for extremely large datasets.
• If splitting files adds too much form complexity, contacting GoCanvas Support is advised.

Additional Notes

• Using multiple large reference files may impact device/app performance due to extra processing required.
• All fields linked to original reference data may need to be set up twice when splitting files.
• Ensuring identical key columns across files is crucial for proper function.

Optimizing Data Sync and Performance

• Schedule reference data updates during off-peak hours to reduce impact on field teams.
• Utilize incremental updates instead of full data reloads to improve sync speed and app responsiveness.
• Regularly audit and remove obsolete or inactive data records to streamline future uploads and maintain accuracy.

Troubleshooting Common Issues

• If dynamic reference data isn’t updating, verify your data source connection and sync status in GoCanvas.
• Check for file format errors or duplicate entries that could prevent imports or updates.
• Reach out to GoCanvas support if issues persist—quicker resolution often comes with detailed examples and error descriptions.

Conclusion: Keep Your Reference Data Seamless

Managing large reference data sets in GoCanvas is essential for digital transformation and operational efficiency. By organizing data, optimizing syncs, and resolving sync challenges promptly, your business empowers teams with reliable insights—helping you stay agile and compliant within your industry.