Cloudformation-coverage-roadmap: Allow use of more than 1 S3 destination buckets

Created on 22 Dec 2020  路  1Comment  路  Source: aws-cloudformation/cloudformation-coverage-roadmap

S3 replication now allows for multiple destination buckets into different regions.

https://aws.amazon.com/blogs/aws/new-amazon-s3-replication-adds-support-for-multiple-destination-buckets/#:~:text=S3%20Replication%20now%20gives%20you,or%20a%20combination%20of%20both

CloudFormation does not yet support the ability to use this feature in conjunction with S3. CloudFormation only allows specifying a single S3 ARN.

When attempting to specify more than 1 distinct destination bucket ARNs, the following error is received:

"Number of distinct destination bucket ARNs cannot exceed 1"

Enabling this functionality would allow for use of cross-region S3 buckets for wider deployments.

enhancement

Most helpful comment

It is supported as of now, but not well documented.

Basically you need to ensure you force rules to use the new Replication Rules V2 schema to support multiple destination buckets. As per https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-add-config.html#replication-backward-compat-considerations V2 schema is forced by specifying the Filter property on each rule.

Once you do this you need to ensure you add a number of configuration properties for each rule as per the example below, and you also need to ensure each Priority is a unique value.

Here is a working example:

Resources:
  PublicBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: foo-sandbox-us-west-2-public-web
      VersioningConfiguration:
        Status: Enabled
      ReplicationConfiguration:
        Role: !GetAtt PublicBucketReplicationRole.Arn
        Rules:
          - Id: ap-southeast-1
            Status: Enabled
            DeleteMarkerReplication:
              Status: Enabled
            Priority: 1
            Filter:
              Prefix: ''
            Destination:
              Bucket: arn:aws:s3:::foo-sandbox-ap-southeast-1-public-web
          - Id: ap-southeast-2
            Status: Enabled
            DeleteMarkerReplication:
              Status: Enabled
            Priority: 2
            Filter:
              Prefix: ''
            Destination:
              Bucket: arn:aws:s3:::foo-sandbox-ap-southeast-2-public-web
  PublicBucketReplicationRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: Allow
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                - s3:GetReplicationConfiguration
                - s3:ListBucket
                Resource: arn:aws:s3:::foo-sandbox-us-west-2-public-web
              - Effect: Allow
                Action:
                - s3:GetObjectVersionForReplication
                - s3:GetObjectVersionAcl
                - s3:GetObjectVersionTagging
                Resource: arn:aws:s3:::foo-sandbox-us-west-2-public-web/*
              - Effect: Allow
                Action:
                - s3:GetObjectVersionTagging
                - s3:ReplicateObject
                - s3:ReplicateDelete
                - s3:ReplicateTags
                Resource: arn:aws:s3:::foo-sandbox-*-public-web/*

>All comments

It is supported as of now, but not well documented.

Basically you need to ensure you force rules to use the new Replication Rules V2 schema to support multiple destination buckets. As per https://docs.aws.amazon.com/AmazonS3/latest/dev/replication-add-config.html#replication-backward-compat-considerations V2 schema is forced by specifying the Filter property on each rule.

Once you do this you need to ensure you add a number of configuration properties for each rule as per the example below, and you also need to ensure each Priority is a unique value.

Here is a working example:

Resources:
  PublicBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: foo-sandbox-us-west-2-public-web
      VersioningConfiguration:
        Status: Enabled
      ReplicationConfiguration:
        Role: !GetAtt PublicBucketReplicationRole.Arn
        Rules:
          - Id: ap-southeast-1
            Status: Enabled
            DeleteMarkerReplication:
              Status: Enabled
            Priority: 1
            Filter:
              Prefix: ''
            Destination:
              Bucket: arn:aws:s3:::foo-sandbox-ap-southeast-1-public-web
          - Id: ap-southeast-2
            Status: Enabled
            DeleteMarkerReplication:
              Status: Enabled
            Priority: 2
            Filter:
              Prefix: ''
            Destination:
              Bucket: arn:aws:s3:::foo-sandbox-ap-southeast-2-public-web
  PublicBucketReplicationRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              Service: s3.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: Allow
          PolicyDocument:
            Statement:
              - Effect: Allow
                Action:
                - s3:GetReplicationConfiguration
                - s3:ListBucket
                Resource: arn:aws:s3:::foo-sandbox-us-west-2-public-web
              - Effect: Allow
                Action:
                - s3:GetObjectVersionForReplication
                - s3:GetObjectVersionAcl
                - s3:GetObjectVersionTagging
                Resource: arn:aws:s3:::foo-sandbox-us-west-2-public-web/*
              - Effect: Allow
                Action:
                - s3:GetObjectVersionTagging
                - s3:ReplicateObject
                - s3:ReplicateDelete
                - s3:ReplicateTags
                Resource: arn:aws:s3:::foo-sandbox-*-public-web/*
Was this page helpful?
0 / 5 - 0 ratings