Skip to content

Scheduling

Importing catalog data

Catalog data is imported using either the standard PUT or POST HTTP method, with personal cluster credentials provided by Voyado. If the catalog import is valid, a 204 OK response is returned. If it is invalid, an error response is returned with details describing the issues in the file. For more information about validation, see Validation. For details about the import request, see Import.

Scheduled imports

Once live, catalog data must be imported on a regular basis. This is typically handled by a program that queues and sends imports at regular intervals, either to fully replace the catalog or to update the catalog incrementally.

Replacing the catalog

Fully replacing the catalog, an import containing all product data, is recommended to be sent on a regular basis for ensuring data consistency.

  • Small databases (fewer than 10,000 products): Imports that replace the catalog can be used exclusively. In this case, imports are recommended to run once per hour.
  • Large databases: Full catalog replacements are still recommended, though at a lower frequency, such as once per day or week. These should be supplemented with smaller, more frequent imports that update specific parts of the catalog.

Updating the catalog

Incrementally updating the catalog can be done through various import operations, allowing both structural changes to product groups and modifications to individual variants, products, or product groups.

Updates are typically triggered by changes in ERP systems, such as price and stock updates, or by product updates from a PIM system. Examples of different types of updates can be found here

Queueing imports

Imports are processed one at a time

Incoming imports will be rejected while another import is being processed. This means that software handling catalog imports, must be set up to run imports one at a time.

Since imports are processed one at a time and for various reasons could fail, all imports should ideally be sent to a queue that aggregate imports, handles retries, and handle errors. A typical process for this is normally responsible for the following or similar tasks:

  • Changes invoked by other systems are placed in a queue
  • Once every e.g. 5 minutes, empty queue and if any changes, aggregate them and send as a single import
  • Log and notify an appropriate person to take action upon errors
  • Remove successfully imported changes from the queue
  • Save unsuccessful import files for easier debugging

Long-running imports

Depending on the size of the import to Elevate, it might take some time. Imports exceeding the maximum duration respond with 202 ACCEPTED and will return an import id that can be used to query the status of the import. The default maximum duration allowed for blocking imports can be overridden by adding the blockingTimeout parameter to the request. The blocking timeout specifies the time to wait after the whole import body has been received. The latest status for an import will be kept for 7 days.

Example

The following example showcases the request flow for an import exceeding the blockingTimeout

Import using blockingTimeout of 1 minute (=PT1M)

curl -i \
-X POST \
-H 'Api-Key: {api-key}' \
-H 'Accept-Encoding: gzip' \
-H 'Content-Type: application/xml' \
"https://{cluster-id}.elevate-api.cloud/api/admin/v3/import/data?blockingTimeout=PT1M&force=false&name=name" \
-T request-body.file

After the blocking timeout, the server hasn't yet processed the file. The server responds with 202 ACCEPTED and the body:

{
    "id": "5537431447739019570",
    "status": "IN_PROGRESS",
    "reason": ""
}

The previously returned import id can now be used to periodically poll the import status.

curl -i \
-X GET \
-H 'Api-Key: {api-key}' \
-H 'Accept-Encoding: gzip' \
"https://{cluster-id}.elevate-api.cloud/api/admin/v3/import/status?id=5537431447739019570"

The import status responds with 200 OK and the status in the body:

{
    "id": "5537431447739019570",
    "status": "SUCCESS",
    "reason": ""
}

Import statuses

Status Description
NOT_FOUND No import found with the supplied id.
WAITING The import is being received, unpackaged, stored or waiting for a previous import to complete.
IN_PROGRESS The import is currently being processed.
SUCCESS The import completed successfully, and the changes have been applied.
FAILURE The import failed, and a reason should be included.
ABORTED The client disconnected while waiting for the import to be processed.

Polling handling

When polling the status of an import, the result fall into three different groups.

  1. The import is being handled, keep polling. Which occurs during WAITING and IN_PROGRESS.
  2. The import completed. Which occurs during SUCCESS and FAILURE Check reason for more information.
  3. The import was not found. Which occurs during NOT_FOUND.

Since ABORTED is raised only when the client closes the connection, it will not be returned in the /v3/import/data result and neither will an import id. Which allows it to be omitted from the polling handling, however, it can be included. The only way to get the ABORTED status is to get the import id from the logs in the apps.

Error handling

All imports are atomic (i.e. either the entire import is processed or nothing will be imported), to make it easier to recover from errors. An error can be caused either by the content being invalid or because of other factors, such as network errors. If an error is received, the import can be safely rerun once the connection is restored or content of the catalog import has been resolved.

A detailed list of import response codes can be found on the specifcation for the import endpoint.

If a retailer experiences persistent network errors they should first contact their network administrator. If the problem seems to be on Voyado's end, please contact Voyado Support.

Temporary network errors

The catalog import software should handle temporary network errors without manual intervention as this is usually done through retries.

Force imports

Potentially harmful imports

Do not override the built in protection in scheduled imports. Overrides should only be made for isolated and manually triggered imports.

Protection from potentially harmful imports is built into Voyado Elevate. If an import looks dangerous, e.g. it removes half of all products in a market it will be rejected with an appropriate error message and code 400. To override the protection and force Elevate to accept such an import, a force=true argument can be added to the post request of the import.

×
Copyright

This online publication is intellectual property of Voyado Lund AB. Its contents can be duplicated in part or whole, provided that a copyright label is visibly located on each copy and the copy is used in conjunction with the product described within this document.

All information found in these documents has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither Voyado Lund AB nor the authors shall be held liable for possible errors or the consequences thereof.

Software and hardware descriptions cited in these documents might be registered trademarks. All trade names are subject to copyright restrictions and may be registered trademarks. Voyado Lund AB essentially adheres to the manufacturer’s spelling. Names of products and trademarks appearing in this document, with or without specific notation, are likewise subject to trademark and trade protection laws and may thus fall under copyright restrictions.

CLOSE