Migrate from AWS S3 storage to Zata.ai S3 storage

Prerequisites

Select any Linux Operating system to perform these processes.

We have chosen the operating system Ubuntu 22.04

Click on the link to Launch Instance.

Setup AWS

For AWS configuration

Click on the AWS console Login

Create a bucket in AWS S3 storage

After logging in to AWS, search for S3 storage service and create a bucket.

To set bucket policy and permission

After creating the bucket apply bucket permission and bucket policy

Click on the created bucket > Permission > Bucket Policy > Edit

Update this policy type to allow modifications to the bucket name, user name, and user account ID in this section.

When applying a bucket policy, ensure you customize the values based on your requirements. Do not copy this policy directly.

To create Access and secret key

AWS Access & Secret Key Pair navigate to the user section

Search for the IAM service and select the existing user to create an access and secret key. If the user does not exist, create the user and assign administrator access permissions.

Verify the user's permission to access the S3 buckets.

To create the Access and Secret Key, go to Security Credentials.

Now create the security credentials to click on the security credentials to create the access key

Before clicking on "Done," first copy the access and secret keys to your notepad.

After performing these processes in AWS, configure the Zata.ai settings.

Setup Zata.ai

For Zata.ai configuration:

• Clink on the zata login page.
• Zata Bucket - See our Bucket Creation Guide
• Zata Access & Secret Key Pair - See our Access Keys Guide
• For more information about the zata.ai

Guide to Move Data From AWS to Zata.ai.

We used Ubuntu 22.04 as the mediator to facilitate the data migration from AWS S3 to Zata.ai S3.

Fro example, on a Ubuntu 22.04 system.

To First Process for AWS

Update && Upgrade the system.

sudo apt-get update -y
sudo apt-get upgrade -y

For migrating data from AWS to Zata.ai, use rclone

Install the configure the rclone

Install and configure the rclone.

sudo apt-get install rclone -y

To configure rclone for AWS, and then configure it in Zata.ai.

Run the following command to configure ' rclone '

sudo rclone config

When you run the command above, follow these steps:

Choose 'n' to create a new remote

Name: AWS (Give the name of the remote)

Type of storage to configure to Choose '4' for Aws s3 compliant storage provider

Choose your S3 provider to Choose '1' AWS s3

Get AWS credentials from runtime (environment variables or EC2/ECS metadata if no env vars). This only applies if access_key_id and secret_access_key are blank. Enter a boolean value (true or false). Press Enter for the default ("false"). Choose a number from below, or type in your value

env_auth: true

Access & Secret key - We have created the above in the AWS section

EndPoint: Select According to region, we have used the Mumbai region ( s3.ap-south-1.amazonaws.com )

Region to connect to. Enter a string value. Choose a number from below, or type in your value

Choose '17' to Asia Pacific(Mumbai)

Endpoint for S3 API. Leave blank if using AWS to use the default endpoint for the region. Enter a string value. Endpoint> s3.ap-south-1.amazonaws.com

Select the default value press enter, then change only the storage class to choose '2'.

Note that this ACL is applied only when creating buckets to choose '4' and set to all default to press Enter.

After completing these processes, simply press Enter at each step.

This is the remote configuration completed after following these steps:

Second process for Zata.ai

The same process is now being applied to Zata.ai.

Choose 'n' to create a new remote.

Name it: zata

Choose '4' for S3 Compliant Storage Provider

Choose'13' for S3 compatible provider

env_auth : true

Access & Secret Key Pair - See our Access Keys Guide

Endpoint: idr01.zata.ai

Ensure you have created buckets in both AWS and Zata.ai before running these commands.

Now complete the configuration to run these commands.

To copy and move your data

rclone copy Aws:(aws_bucket_name) Zata:(zata_bucket_name) --s3-region ap-south-1 --progress

Now, return to both AWS and zata.ai to check the status of buckets and details of the data.