Copy S3 Buckets & data between AWS Accounts using IAM Roles & Cloudshell

➤ Transferring data between AWS accounts, especially S3 buckets, is a common task that can be necessary for data migration, collaboration, or backup purposes. In this guide, I will walk you through the process of copying an S3 bucket from one AWS account to another using IAM roles and the AWS CLI.

Step 1: Create IAM Roles in Both AWS Accounts

➤ To facilitate secure data transfer between AWS accounts, you need to create IAM roles in both the source and destination accounts. These roles will grant the necessary permissions to access and copy the S3 bucket contents.

1.1. Create an IAM Role in the Destination Account

1- Log in to the AWS Management Console of the destination account.

2- Navigate to the IAM Console.

3- Click on Create Role.

4- In the Select Trusted Entity section, choose AWS Account

5- Select Another AWS account and enter the AWS Account ID of the source account.

6- Attach S3 Full Access Policy:

➤ Add the AmazonS3FullAccess policy to this role. This will grant the necessary permissions to perform actions on S3.

7- Click Next, provide a Role Name (e.g., S3CopyDestinationRole), and then click Create Role.

1.2. Create an IAM Role in the Source Account

1- Log in to the AWS Management Console of the source account.

2- Navigate to the IAM Console.

3- Click on Create Role.

4- In the Select Trusted Entity section, choose AWS Account

5- Select Another AWS account and enter the AWS Account ID of the destination account.

6- Attach S3 Full Access Policy:

➤ Add the AmazonS3FullAccess policy to this role. This will grant the necessary permissions to perform actions on S3.

7- Click Next, provide a Role Name (e.g., S3CopyDestinationRole), and then click Create Role.

Step 2: Update the S3 Bucket Policies

To enable cross-account access for S3 bucket data transfer, you need to update the bucket policies in both the source and destination accounts.

2.1. Update the Destination Bucket Policy

1- Go to the S3 Console of the destination account.

2- Open the destination bucket that will receive the copied objects.

3- Go to the Permissions tab.

4- Scroll down to the Bucket Policy section and click on Edit.

5- Add the following bucket policy, this policy copied from this github repo replacing placeholders with actual ARNs:

➤ Replace Source-Account-ID with the account ID of the source account.

➤ Replace Destination-Bucket-Name with the name of your destination bucket.

2.2. Update the Source Bucket Policy

1- Go to the S3 Console of the source account.

2- Open the source bucket that contains the objects you want to copy.

3- Go to the Permissions tab.

4- Scroll down to the Bucket Policy section and click on Edit.

5- Add the following bucket policy, this policy copied from this github repo

replacing placeholders with actual ARNs:

➤ Replace Destination-Account-ID with the account ID of the destination account.

➤ Replace Source-Bucket-Name with the name of your source bucket.

Step 3: Copy Data Between S3 Buckets

Now that the IAM roles and bucket policies are configured, you can copy the data using the AWS CLI.

3.1. Run the AWS CLI Copy Command

1- Log in to the AWS Management Console of the destination account where you have configured the IAM role.

2- Open a terminal or command prompt or Cloudshell on AWS console.

3- Run the following AWS CLI command to copy data from the source bucket to the destination bucket:

4- Using this step your user should have permission to run & execute this script.

 aws s3 cp --recursive s3://<Source-Bucket-Name>/ s3://<Destination-Bucket-Name>/

➤ Replace Source-Bucket-Name with the name of your source bucket.

➤ Replace Destination-Bucket-Name with the name of your destination bucket.

This command will recursively copy all objects from the source bucket to the destination bucket.