How to copy backups from one AWS Backup vault to another

I recently found myself wanting to copy all backups from one AWS Backup vault to another. The reason was that I wanted to move backups from a vault that was about to be deleted to a new vault. The AWS console provides a way to copy backups between vaults, but it's a manual process that has to be initiated for each backup individually. This is not feasible if you have a large number of backups to copy.

Instead, I decided to use the AWS CLI to automate the process. In this blog post, I'll show you how you can copy backups between different AWS Backup vaults using the AWS CLI.

Prerequisites

Before you can copy backups between different AWS Backup vaults, you need to have the following prerequisites in place:

1. AWS CLI: Make sure you have the AWS CLI installed on your machine. You can find installation instructions here. If you are on macOS, you can install the AWS CLI using Homebrew with brew install awscli.
2. AWS credentials: Make sure you have AWS credentials configured on your machine. You can do this by running aws configure and following the prompts to enter your AWS Access Key ID, Secret Access Key, and default region.
3. jq command-line JSON processor: You can install jq on macOS using Homebrew with brew install jq or on Linux using your package manager.
4. ARN of the IAM role: You need the ARN of an IAM role that has permissions to copy backups between vaults. When you manually copy a backup between vaults in the AWS console, the role will be created for you. You can find the ARN of the role in the AWS console under IAM > Roles. The role name will be AWSBackupDefaultServiceRole or similar.
5. Name of the source vault: You need the name of the source vault from which you want to copy the backups. You can find the name of the vault in the AWS console under AWS Backup > Backup vaults.
6. ARN of the destination vault: You need the ARN of the destination vault to which you want to copy the backups. You can find the ARN of the vault in the AWS console after clicking on the vault name. The ARN will look like arn:aws:backup:us-west-2:123456789012:backup-vault:MyDestinationVault.

Copying backups between vaults

To copy backups between different AWS Backup vaults, you can use the following script. Replace the placeholders with your own values:

#!/bin/bash

# This script copies all recovery points from one backup vault to another. It
# is useful when you want to migrate from one vault to another or consolidate
# backups from multiple accounts.
# Source: https://www.smartinary.com/blog/aws-backup-copy-recovery-points-between-vaults/

# Configuration (change these values)
SOURCE_VAULT="MySourceVault"
TARGET_VAULT_ARN="arn:aws:backup:us-west-2:123456789012:backup-vault:MyDestinationVault"
IAM_ROLE_ARN="arn:aws:iam::198378955029:role/service-role/AWSBackupDefaultServiceRole"
# End of configuration

# List all recovery points in the source vault and store the ARN of each recovery point
RECOVERY_POINTS=$(aws backup list-recovery-points-by-backup-vault --backup-vault-name "$SOURCE_VAULT" | jq -r '.RecoveryPoints[].RecoveryPointArn')

# Check if we got any recovery points
if [ -z "$RECOVERY_POINTS" ]; then
    echo "No recovery points found in the source vault."
    exit 1
fi

# Generate an idempotency token based on the current timestamp
TOKEN="copy-$(date +%s)"

# Copy each recovery point to the target vault
for RECOVERY_POINT_ARN in $RECOVERY_POINTS; do
    echo "Copying recovery point $RECOVERY_POINT_ARN to $TARGET_VAULT_ARN with token $TOKEN"
    SUCCESS=0
    while [ $SUCCESS -eq 0 ]; do
        RESULT=$(aws backup start-copy-job --recovery-point-arn "$RECOVERY_POINT_ARN" --source-backup-vault-name "$SOURCE_VAULT" --destination-backup-vault-arn "$TARGET_VAULT_ARN" --iam-role-arn "$IAM_ROLE_ARN" --idempotency-token "$TOKEN" 2>&1)
        if [ $? -eq 0 ]; then
            echo "Copy job initiated successfully!"
            SUCCESS=1
        else
            if echo "$RESULT" | grep -q 'LimitExceededException'; then
                echo "LimitExceededException: waiting 60 seconds before retrying"
                sleep 60
            elif echo "$RESULT" | grep -q 'ServiceUnavailableException'; then
                echo "ServiceUnavailableException: waiting 60 seconds before retrying"
                sleep 60
            elif echo "$RESULT" | grep -q 'NotFoundException'; then
                echo "NotFoundException: skipping this recovery point"
                SUCCESS=1
            else
                echo "Error: $RESULT"
                exit 1
            fi
        fi
    done
done

echo "All recovery points have been initiated for copying."

This script will copy all recovery points from the source vault to the target vault. It uses the aws backup start-copy-job command to initiate the copy job for each recovery point. The script handles errors such as LimitExceededException, ServiceUnavailableException, and NotFoundException by retrying the operation or skipping the recovery point.

Keep in mind that AWS Backup runs copy jobs asynchronously, so it may take some time for all recovery points to be copied. You can monitor the progress in the AWS Backup console under Jobs > Copy jobs. Make sure to check the status of the copy jobs to ensure that they have completed successfully, especially if you are planning to delete the source vault.

Conclusion

Copying backups between different AWS Backup vaults can be a useful operation when you need to migrate backups from one vault to another or consolidate backups from multiple accounts into a single account. By using the AWS CLI and the script provided in this blog post, you can automate the process and save time when dealing with a large number of recovery points.