File Size and Transfer Restrictions

Upload Limits

If you do a WebDAV push into a Salesforce B2C Commerce instance, the upload limit is 100 MB. But how much you can upload also varies by the speed and quality of your connection. If it takes too long to transfer the file, the connection might time out. Salesforce recommends zipping files for upload. If you have difficulty uploading a file under 100 MB, contact B2C Commerce customer support.

Download Limits

File sizes must be under 200 MB for FTP or WebDAV if you download into a file. The file sizes if you download into a string are much smaller.

FTPClient Size Limitations

dw.net.FTPClient now enforces size limits for file fetches. These size limits can be passed in as arguments for dw.net.FTPClient.get() or they can be defaulted.

- The default size for a get operation that returns a string is dw.net.FTPClient.DEFAULT_GET_STRING_SIZE (2Mb).

- The default for a get operation that creates a local file is dw.net.FTPClient.DEFAULT_GET_FILE_SIZE (10Mb).

- The maximum download size for a string is dw.net.FTPClient.MAX_GET_STRING_SIZE (10Mb) and for a file is dw.net.FTPClient.MAX_GET_FILE_SIZE(200Mb).

Note: These are the size limitations for compressed files, so the uncompressed files can be much larger.

HTTPClient Size Limitations

The HTTP client doesn't have a file size limitation.

Restrictions on External XML Entities and the DTD Declaration

For all XML imports (catalog, customer, inventory, etc) you can't use external XML entities and the DTD declaration. They have been deactivated (as of 13.3.3) in order to prevent XML External Entities (XXE) attacks.

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE catalog [ <!ENTITY remote SYSTEM "http://remote.apt.vsecurity.org/"> ]>

Compressed Feeds

XML file sizes can be reduced significantly through compression. Compressed feeds might help overcome upload size limits and shorten upload/download times on slow network links.

The standard import support files are compressed with the gzip algorithm natively. An explicit gunzip at the instance is unnecessary. Simply provide a file name such as catalog.xml.gz to the import pipelet or select a gzipped file (extension must be .gz) in the Business Manager.

Feeds can be zipped and unzipped (zip compression, not gzip compression) on a single file basis within Import/Export sections in the Business Manager. Also, the Business Manager automatically unzips .zip files during an upload. You can programmatically zip/unzip files via the APIs dw.io.File.zip and dw.io.File.unzip.

Static Files

The persistent objects in the catalog and content library are supplemented with static files, typically product images, that are stored in the instance file system. The XML feeds contain database objects only. To export/import or download/upload static files, you can use the WebDAV interface. B2C Commerce also lets you zip/unzip multiple files to reduce the number of download/upload operations via the APIs dw.io.File.zip and dw.io.File.unzip.

Size and Runtime Limits

Imports and exports are designed to deal with an arbitrary number of objects in a feed. Some customers work with feeds containing many hundreds of megabytes. The import/export processes support large data sets via chunking techniques, where only the currently processed items exist in memory. While import/export files don't have size limits, size limits might exist for uploading to an instance.

The system imposes a timeout for requests to a B2C Commerce instance. The web server closes the connection if the application server doesn't respond within 5 minutes. Large imports or exports, however, typically run longer than 5 minutes. To avoid a request timeout, insert import/export pipelets in a job pipeline and execute it asynchronously. There are no run-time limits for job pipelines.

Related Links

File Size and Transfer Restrictions

Using WebDAV

Using FTP

Using HTTPS