How are massive dataset uploads managed?

Data Loaders FAQ #6: How are massive dataset uploads managed by Data Loaders?


The maximum number of records that can be uploaded or downloaded between Excel and QAD using Data Loaders will vary based on two factors.

First of all, your QAD server settings, particularly the -L progress server startup parameter, determine the number of locks. The larger the number, the more records can be uploaded per batch. Increasing the -L value will allow the user to upload more records, but it will also consume more of the server’s resources. This can be a good solution for some. For others it will not be viable. Please coordinate with your DBA.

Secondly, the number of locks does not necessarily correspond to the number of records in the Excel spreadsheet. In most applications, one record that a user sees in Excel may update many records in the QAD database. For example, the 1.10.1.1 QAD standard program updates many records in a quite a few different tables for one price list record that the user sees in Excel.

What happens when Data Loaders hits the limit of the number of records that can be loaded to QAD? It is a completely safe process and harmless to the QAD application. When the limit is reached, the QAD server disconnects the Excel user and the progress session is terminated. The user will receive a “time out” message and none of the records in the batch will be updated to QAD.

As general recommendation, we suggest that you run massive updates to QAD in hours when fewer users are logged in to QAD and less batch processes are running. This will allow more server resources to be allocated to excel updates.

For additional recommendations and tips to help you better manage massive uploads, please read the following article: http://www.32soft.com/updating-large-datasets-in-qad-with-excel-loaders/

We hope this email and the tips and recommendations in the article answer your questions about how Data Loaders handles massive uploads. If you have further questions, you can visit our FAQs page or Product Demos on our website, or feel free to contact us with your questions at 1-877-348-5432. We look forward to hearing from you.

Click to go back.