Migrate from AWS S3 storage to Zata.ai S3 storage
Last updated
Last updated
Select any Linux Operating system to perform these processes.
We have chosen the operating system Ubuntu 22.04
Click on the link to Launch Instance.
For AWS configuration
Click on the AWS console Login
After logging in to AWS, search for S3 storage service and create a bucket.
After creating the bucket apply bucket permission and bucket policy
Click on the created bucket > Permission > Bucket Policy > Edit
Update this policy type to allow modifications to the bucket name, user name, and user account ID in this section.
When applying a bucket policy, ensure you customize the values based on your requirements. Do not copy this policy directly.
AWS Access & Secret Key Pair navigate to the user section
Search for the IAM service and select the existing user to create an access and secret key. If the user does not exist, create the user and assign administrator access permissions.
Verify the user's permission to access the S3 buckets.
To create the Access and Secret Key, go to Security Credentials.
Now create the security credentials to click on the security credentials to create the access key
Before clicking on "Done," first copy the access and secret keys to your notepad.
After performing these processes in AWS, configure the Zata.ai settings.
For Zata.ai configuration:
Clink on the zata login page.
Zata Bucket - See our Bucket Creation Guide
Zata Access & Secret Key Pair - See our Access Keys Guide
For more information about the zata.ai
We used Ubuntu 22.04 as the mediator to facilitate the data migration from AWS S3 to Zata.ai S3.
Fro example, on a Ubuntu 22.04 system.
Update && Upgrade the system.
For migrating data from AWS to Zata.ai, use rclone
Install and configure the rclone.
To configure rclone for AWS, and then configure it in Zata.ai.
Run the following command to configure ' rclone '
When you run the command above, follow these steps:
Choose 'n' to create a new remote
Name: AWS (Give the name of the remote)
Type of storage to configure to Choose '4' for Aws s3 compliant storage provider
Choose your S3 provider to Choose '1' AWS s3
Get AWS credentials from runtime (environment variables or EC2/ECS metadata if no env vars). This only applies if access_key_id and secret_access_key are blank. Enter a boolean value (true or false). Press Enter for the default ("false"). Choose a number from below, or type in your value
env_auth: true
Access & Secret key - We have created the above in the AWS section
EndPoint: Select According to region, we have used the Mumbai region ( s3.ap-south-1.amazonaws.com )
Region to connect to. Enter a string value. Choose a number from below, or type in your value
Choose '17' to Asia Pacific(Mumbai)
Endpoint for S3 API. Leave blank if using AWS to use the default endpoint for the region. Enter a string value. Endpoint> s3.ap-south-1.amazonaws.com
Select the default value press enter, then change only the storage class to choose '2'.
Note that this ACL is applied only when creating buckets to choose '4' and set to all default to press Enter.
After completing these processes, simply press Enter at each step.
This is the remote configuration completed after following these steps:
The same process is now being applied to Zata.ai.
Choose 'n' to create a new remote.
Name it: zata
Choose '4' for S3 Compliant Storage Provider
Choose'13' for S3 compatible provider
env_auth : true
Access & Secret Key Pair - See our Access Keys Guide
Endpoint: idr01.zata.ai
Ensure you have created buckets in both AWS and Zata.ai before running these commands.
Now complete the configuration to run these commands.
Now, return to both AWS and zata.ai to check the status of buckets and details of the data.
