Python (boto3)
Upload, download and share files from Python with boto3.
boto3 is the AWS SDK for Python. Point it at the Zata endpoint, and you can use the normal S3 calls to work with your buckets from Python code.
Before you start
- A bucket. See Create a bucket.
- An access key and secret key from the same region as the bucket. See Access keys.
- Python 3 with boto3:
pip install boto3.
Connection settings
| Setting | Value |
|---|---|
endpoint_url | https://idr01.zata.ai for Indore, https://bom01.zata.ai for Mumbai |
region_name | us-east-1 |
aws_access_key_id / aws_secret_access_key | from Access Keys in the Zata console. The key must be from the same region as the bucket. |
addressing_style | path |
request_checksum_calculation / response_checksum_validation | when_required |
Create the client
Keep your keys out of the code. Put them in environment variables:
export ZATA_ACCESS_KEY=YOUR_ACCESS_KEY
export ZATA_SECRET_KEY=YOUR_SECRET_KEY
Then create the client. For a bucket in Mumbai, use https://bom01.zata.ai.
import os
import boto3
from botocore.config import Config
s3 = boto3.client(
"s3",
endpoint_url="https://idr01.zata.ai",
region_name="us-east-1",
aws_access_key_id=os.environ["ZATA_ACCESS_KEY"],
aws_secret_access_key=os.environ["ZATA_SECRET_KEY"],
config=Config(
signature_version="s3v4",
s3={"addressing_style": "path"},
request_checksum_calculation="when_required",
response_checksum_validation="when_required",
),
)
The two checksum settings are needed on botocore 1.36 and newer: without them, large uploads fail with XAmzContentSHA256Mismatch. On older botocore versions, leave those two lines out.
Upload and list files
s3.upload_file("myfile.txt", "my-bucket", "folder/myfile.txt")
for obj in s3.list_objects_v2(Bucket="my-bucket", Prefix="folder/").get("Contents", []):
print(obj["Key"], obj["Size"])
upload_file uploads large files in parts by itself.
Download a file
s3.download_file("my-bucket", "folder/myfile.txt", "myfile-copy.txt")
List your buckets
for bucket in s3.list_buckets()["Buckets"]:
print(bucket["Name"])
A sub-user's key gets an empty list here, even for buckets it can use. Use the bucket name directly.
Make a download or upload link
A presigned URL lets someone download or upload one file without keys, until the link expires. ExpiresIn is in seconds.
download_url = s3.generate_presigned_url(
"get_object",
Params={"Bucket": "my-bucket", "Key": "folder/myfile.txt"},
ExpiresIn=3600,
)
upload_url = s3.generate_presigned_url(
"put_object",
Params={"Bucket": "my-bucket", "Key": "upload.txt"},
ExpiresIn=3600,
)
Anyone can then upload with the upload link, for example with curl:
curl -X PUT --upload-file upload.txt "UPLOAD_URL"
- If you add a
ContentTypetoParams(for example"ContentType": "text/plain"), the upload must send exactly that header (-H "Content-Type: text/plain"), or Zata answers403. Without aContentTypein the link, send noContent-Typeheader. - A link stops working when it expires, or when the key that signed it is deleted or turned off.
- Anyone who has the link can use it until it expires. Share it only with the intended person.
Troubleshooting
| Problem | Fix |
|---|---|
Large uploads fail with XAmzContentSHA256Mismatch; small files work | Add the two when_required settings to Config. Or set the environment variables AWS_REQUEST_CHECKSUM_CALCULATION=when_required and AWS_RESPONSE_CHECKSUM_VALIDATION=when_required. |
InvalidAccessKeyId | endpoint_url is missing (requests go to AWS), or the key is from the other region. |
NoSuchBucket for a bucket you can see in the console | Use the endpoint of the region the bucket is in. |
403 when using an upload link | The Content-Type header differs from the one in the link. |
TypeError about request_checksum_calculation | Your botocore is older than 1.36. Update boto3, or leave the two checksum lines out. |
Never use verify=False: Zata's certificates are valid. For other errors, see Large uploads and Wrong endpoint or region.