Large uploads fail

Help

This page helps when small files upload but large ones fail, when an upload stalls or keeps retrying, or when a large file's ETag does not match its MD5. It also covers uploads rejected because of an encryption or storage class option.

Large files are uploaded in parts (multipart upload). Most tools switch to multipart by themselves — for example aws s3 cp for files over 8 MB.

Quick fixes

You see	Most likely cause	Fix
`XAmzContentSHA256Mismatch`, `BadDigest` or a checksum error — only on large files	New AWS CLI and SDK versions send default checksums that Zata rejects on multipart uploads	Set checksums to "when required"
AWS SDK for Java v2: `XAmzContentSHA256Mismatch` on every upload	The same default checksums (SDK 2.30 and newer)	Set checksums to "when required"
AWS CLI: `argument of type 'NoneType' is not a container or iterable` on large files	Usually the checksum problem above	Set the checksum settings, or run with `--debug`
`EntityTooLarge`	A file over 5 GB sent in one request, without multipart	Use multipart
`EntityTooSmall`	A part other than the last is smaller than 5 MB	Use bigger parts
`InvalidPart`	The part list sent at the end does not match the uploaded parts (ETag or part number)	Build the list again from the parts Zata received
`NoSuchUpload`	Wrong upload ID, or the upload was already completed or cancelled	Start a new upload
Upload is slow, stalls or times out	Too many parallel parts for the connection	Lower concurrency, raise retries
The ETag of a large file is not its MD5	Normal for multipart uploads — not corruption	Check size and checksum instead
The bucket uses more storage than its files	Unfinished multipart uploads	Clean them up
`InvalidRequest` with encryption turned on in the tool	Zata does not accept server-side encryption requests	Turn the tool's encryption option off
`InvalidArgument` with a storage class set	Zata has one storage class	Leave the storage class empty or `STANDARD`

Checksum errors with new AWS tools

AWS CLI and AWS SDK versions released from January 2025 add checksums to uploads by default. Zata rejects them on multipart uploads: small files upload, large files fail with XAmzContentSHA256Mismatch. This is the usual cause when a tool "worked before an upgrade", or when rclone works but another tool fails.

Tool	Affected versions	What fails
AWS CLI	v2.23 and newer	large (multipart) uploads
boto3 / botocore	1.36 and newer	large (multipart) uploads
AWS SDK for JavaScript v3	3.729 and newer	large (multipart) uploads
AWS SDK for Java v2	2.30 and newer	every upload, even small files

rclone is not affected, and neither are versions older than the ones above.

Fix: tell the tool to calculate checksums only when they are required.

aws configure set request_checksum_calculation when_required --profile zata
aws configure set response_checksum_validation when_required --profile zata

This adds two lines to the [profile zata] section of ~/.aws/config:

request_checksum_calculation = when_required
response_checksum_validation = when_required

from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url="https://idr01.zata.ai",
    region_name="us-east-1",
    config=Config(
        signature_version="s3v4",
        s3={"addressing_style": "path"},
        request_checksum_calculation="when_required",
        response_checksum_validation="when_required",
    ),
)

const s3 = new S3Client({
  endpoint: "https://idr01.zata.ai",
  region: "us-east-1",
  forcePathStyle: true,
  requestChecksumCalculation: "WHEN_REQUIRED",
  responseChecksumValidation: "WHEN_REQUIRED",
});

S3Client s3 = S3Client.builder()
    .endpointOverride(URI.create("https://idr01.zata.ai"))
    .region(Region.US_EAST_1)
    .forcePathStyle(true)
    .requestChecksumCalculation(RequestChecksumCalculation.WHEN_REQUIRED)
    .responseChecksumValidation(ResponseChecksumValidation.WHEN_REQUIRED)
    .build();

client := s3.NewFromConfig(cfg, func(o *s3.Options) {
    o.BaseEndpoint = aws.String("https://idr01.zata.ai")
    o.UsePathStyle = true
    o.RequestChecksumCalculation = aws.RequestChecksumCalculationWhenRequired
    o.ResponseChecksumValidation = aws.ResponseChecksumValidationWhenRequired
})

Go apps also need the Accept-Encoding fix from Signature errors.

For any app built on an AWS SDK, set two environment variables before you start it:

export AWS_REQUEST_CHECKSUM_CALCULATION=when_required
export AWS_RESPONSE_CHECKSUM_VALIDATION=when_required

On Windows (PowerShell):

$env:AWS_REQUEST_CHECKSUM_CALCULATION="when_required"
$env:AWS_RESPONSE_CHECKSUM_VALIDATION="when_required"

In --debug output, these default checksums show up as x-amz-checksum-crc64nvme or x-amz-checksum-crc32 headers (often with x-amz-sdk-checksum-algorithm or x-amz-trailer). The x-amz-content-sha256 header is part of normal signing and is not the problem.

Full setup for each tool: AWS CLI, Python, JavaScript, Java, Go.

Size and part limits

Limit	Value	Error when exceeded
One upload without multipart	5 GB at most	`EntityTooLarge`
Size of each part, except the last	5 MB at least	`EntityTooSmall`
Number of parts in one upload	10,000 at most	—

A multipart upload can be much larger than 5 GB. Parts may be uploaded in any order.

EntityTooLarge: use a command that uploads in parts. For example, use aws s3 cp instead of aws s3api put-object, which always sends the file in one request.
EntityTooSmall: raise the part size in your tool or code.
If the final "complete" step fails, Zata removes that upload. Start the upload again.

Upload stalls or keeps retrying

First upload one small and one large file with the same tool. If only the large one fails with an error code, use the table at the top. If there is no error code but the upload is slow or times out, upload fewer parts at the same time and allow more retries:

AWS CLI
rclone

aws configure set s3.max_concurrent_requests 4 --profile zata

rclone copyto /path/to/bigfile.iso zata:my-bucket/bigfile.iso --transfers 2 --s3-upload-concurrency 2 --retries 5 --low-level-retries 20 -P

On a fast but unstable link, bigger parts mean fewer requests: try --s3-chunk-size 64M. Each upload uses about chunk size × concurrency of memory. If an rclone upload stops half way, run the same command again: files already in the bucket are skipped.

Check that a large file arrived intact

For a multipart upload, the ETag is not the file's MD5. It ends in -N, where N is the number of parts, so a different ETag does not mean the file is damaged. Compare the size and a checksum instead:

aws s3api head-object --bucket my-bucket --key bigfile.iso --profile zata --query ContentLength
aws s3 cp s3://my-bucket/bigfile.iso ./check.iso --profile zata
sha256sum bigfile.iso check.iso

The size must equal the local file's size, and the two SHA-256 sums must match. With rclone, rclone check ./folder zata:my-bucket/folder --one-way --download compares the content of files uploaded in parts.

Clean up unfinished uploads

An upload that was never completed or cancelled keeps its parts, and they keep using storage. List them and cancel the ones you do not need:

aws s3api list-multipart-uploads --bucket my-bucket --profile zata
aws s3api abort-multipart-upload --bucket my-bucket --key KEY --upload-id UPLOAD_ID --profile zata

Take KEY and UPLOAD_ID from the list. Cancelling deletes the parts uploaded so far. A sub-user key may get AccessDenied on the list — use the account owner's key.

To clean up automatically, add a lifecycle rule with Discard unfinished uploads. See Lifecycle rules.

Test a multipart upload step by step

If another app cannot upload large files, do each multipart step by hand with the AWS CLI. The step that fails shows the problem.

Make a test file in parts

Every part except the last must be at least 5 MB (Linux or macOS):

head -c 12M /dev/urandom > bigfile.bin
split -b 5M -d bigfile.bin part-

This makes part-00 and part-01 (5 MB each) and part-02 (2 MB).

Start the upload

UPLOAD_ID=$(aws s3api create-multipart-upload --bucket my-bucket --key test/bigfile.bin --query UploadId --output text --profile zata)
echo "$UPLOAD_ID"

Upload the parts

aws s3api upload-part --bucket my-bucket --key test/bigfile.bin --part-number 1 --body part-00 --upload-id "$UPLOAD_ID" --profile zata
aws s3api upload-part --bucket my-bucket --key test/bigfile.bin --part-number 2 --body part-01 --upload-id "$UPLOAD_ID" --profile zata
aws s3api upload-part --bucket my-bucket --key test/bigfile.bin --part-number 3 --body part-02 --upload-id "$UPLOAD_ID" --profile zata

List the parts Zata received

aws s3api list-parts --bucket my-bucket --key test/bigfile.bin --upload-id "$UPLOAD_ID" --query '{Parts: Parts[].{PartNumber: PartNumber, ETag: ETag}}' --output json --profile zata > parts.json

Complete the upload

aws s3api complete-multipart-upload --bucket my-bucket --key test/bigfile.bin --upload-id "$UPLOAD_ID" --multipart-upload file://parts.json --profile zata

Step	Error	Meaning
Upload the parts	`XAmzContentSHA256Mismatch`	The default checksums — set them to "when required"
Complete	`EntityTooSmall`	A part other than the last is under 5 MB
Complete	`InvalidPart`	An ETag or part number in `parts.json` does not match — run the list step again
Any step	`NoSuchUpload`	Wrong upload ID, or the upload was already completed or cancelled
Any step	`AccessDenied`	The key is not allowed this action on the bucket

If these steps work with the AWS CLI but your app still fails, the app most likely sends the default checksums. Set its checksum option to "when required", or use the environment variables in the Any app tab above.

Options to turn off in your tool

Some tool options ask for S3 features that Zata does not offer. Every upload fails until you turn them off:

Error	Option	What to do
`InvalidRequest`	Server-side encryption (SSE-S3 / `AES256`, SSE-C, SSE-KMS)	Turn it off. To encrypt your data, use a tool that encrypts before upload, such as an rclone crypt remote, restic or Duplicati
`InvalidArgument`	A storage class such as `GLACIER`, `DEEP_ARCHIVE` or `STANDARD_IA`	Leave the storage class empty, or set `STANDARD`
`NotImplemented`	Another S3 feature Zata does not offer	Turn that option off

See the real error

Zata's error responses contain an error code but no message text, so look for the code. The debug output contains your access key ID — remove it before you share the output.

Tool	Run	Look for
AWS CLI	add `--debug` to the command	`<Code>` in the `Response body` line, and `x-amz-checksum-` headers in the request
rclone	add `-vv --dump headers`	the HTTP status and `api error` with the code in the last line
s3fs	mount with `-f -o dbglevel=info`	the failing request and its HTTP response code
boto3	catch `ClientError`	`e.response["Error"]["Code"]`

Still stuck?

Contact Zata support (see Get support) and include:

the tool and its version (aws --version, rclone version, or your SDK version)
the endpoint you use and the region of the bucket (Indore or Mumbai)
the file size, and the part size and concurrency if you set them
the exact error code, and the RequestId from the error response
the debug output from above, with access keys removed

Quick fixes​

Checksum errors with new AWS tools​

Size and part limits​

Upload stalls or keeps retrying​

Check that a large file arrived intact​

Clean up unfinished uploads​

Test a multipart upload step by step​

Options to turn off in your tool​

See the real error​

Still stuck?​