Terraform-provider-aws: Cyclic dependency error when creating buckets with two way replication

[hashibot]

_This issue was originally opened by @chetandeshmukh as hashicorp/terraform#14295. It was migrated here as part of the provider split. The original body of the issue is below._

---

provider "aws" {
    alias   = "east"
    region  = "${var.aws_region_east}"
    profile = "${var.profile}"
}

provider "aws" {
    alias   = "west"
    region  = "${var.aws_region_west}"
    profile = "${var.profile}"
}

resource "aws_s3_bucket" "bucket_west" {
    provider = "aws.west"
    bucket   = "${var.bucket_west}"
    region   = "${var.aws_region_west}"
    acl      = "private"

    versioning {
    enabled = true
    }
    replication_configuration {
        role = "${data.aws_iam_role.cove_iam_role.arn}"
        rules {
            id     = "bucket_west"
            prefix = ""
            status = "Enabled"

            destination {
            bucket        = "${aws_s3_bucket.bucket_east.arn}"
            storage_class = "STANDARD"
            }
        }
    }
}

resource "aws_s3_bucket" "bucket_east" {
    provider = "aws.east"
    bucket   = "${var.bucket_east}"
    region   = "${var.aws_region_east}"
    acl      = "private"

    versioning {
    enabled = true
    }

    replication_configuration {
    role = "${data.aws_iam_role.cove_iam_role.arn}"
        rules {
            id     = "bucket_east"
            prefix = ""
            status = "Enabled"

            destination {
                bucket        = "${aws_s3_bucket.bucket_west.arn}"
                storage_class = "STANDARD"
            }
        }
    }
}

[jmdots]

Any word on this? I too have a need for S3 bidirectional replication via Terraform.

[apparentlymart]

This sort of thing is historically rather tricky to do in Terraform, but we have some patterns.

Probably the most straightforward way would be for the replication configuration to become a separate resource that refers to both buckets, which I think is theoretically possible since replication configuration seems to be a separate endpoint in the S3 API:

resource "aws_s3_bucket" "bucket_west" {
    provider = "aws.west"
    bucket   = "${var.bucket_west}"
    region   = "${var.aws_region_west}"
    acl      = "private"

    versioning {
    enabled = true
    }
}

resource "aws_s3_bucket" "bucket_east" {
    provider = "aws.east"
    bucket   = "${var.bucket_east}"
    region   = "${var.aws_region_east}"
    acl      = "private"

    versioning {
    enabled = true
    }
}

resource "aws_bucket_replication" "eastward" {
  provider = "aws.west"
  source_bucket_name = "${aws_s3_bucket.bucket_west.name}"
  role = "${data.aws_iam_role.cove_iam_role.arn}"
  rules {
    id     = "bucket_west"
    prefix = ""
    status = "Enabled"

    destination {
      bucket        = "${aws_s3_bucket.bucket_east.arn}"
      storage_class = "STANDARD"
    }
  }
}

resource "aws_bucket_replication" "westward" {
  provider = "aws.east"
  source_bucket_name = "${aws_s3_bucket.bucket_east.name}"
  role = "${data.aws_iam_role.cove_iam_role.arn}"
  rules {
    id     = "bucket_west"
    prefix = ""
    status = "Enabled"

    destination {
      bucket        = "${aws_s3_bucket.bucket_west.arn}"
      storage_class = "STANDARD"
    }
  }
}

By splitting this out, the cycle is avoided and instead the replication resources just both depend on both of the buckets.

Unfortunately implementing this separate resource is not straightforward -- it'd require some care to avoid breaking compatibility with the old inline form -- so it's not something we'll be able to jump on immediately :confounded: but if someone else has the time and motivation to work on it we could perhaps work with them to figure out the backward compatibility path and then review a PR.

[mkohn]

Not the best solution but one that works is to manually build the arn into the replication, this will break the cycle issue.

Since S3 is an old service, its only arn:aws:s3:::my_corporate_bucket.

Here is a snippet of my working code.

resource "aws_s3_bucket" "s3-primary" {
  provider = "aws.source_region"
  bucket   = "{var.source_bucket}"

  versioning {
    enabled = true
  }

  replication_configuration {
    role = "${aws_iam_role.replication_role_source_to_dr.arn}"

    rules {
      id     = "replicate_all"
      prefix = ""
      status = "Enabled"

      destination {
        bucket        = "arn:aws:s3:::{var.dest_bucket}"
        storage_class = "STANDARD"
      }
    }
  }
}

resource "aws_s3_bucket" "s3-dr" {
  provider = "aws.dr_region"
  bucket   = "{var.dest_bucket}"

  versioning {
    enabled = true
  }


  replication_configuration {
    role = "${aws_iam_role.replication_role_dr_to_source.arn}"

    rules {
      id     = "replicate_all"
      prefix = ""
      status = "Enabled"

      destination {
        bucket        = "arn:aws:s3:::{var.source_bucket}"
        storage_class = "STANDARD"
      }
    }
  }
}

[CyrilDevOps]

Any news on that ? I had to fake the ARN too, and I don't like that.

[CyrilDevOps]

Today I was creating two new buckets with bidirectionnal replication, but the the fake arn didn't work,
i hit some timing issue, sometime I get the error

• aws_s3_bucket.s3-dr: Error putting S3 replication configuration: InvalidRequest: Destination bucket must exist.
  or
• aws_s3_bucket.s3-dr: Error putting S3 replication configuration: InvalidRequest: Destination bucket must have versioning enabled.

I had to comment the replication block, create my two buckets, then uncomment the
replication bloick and re-plan and apply.

We need the replication by itself outside of the aws_s3_bucket so terraform can manage the dependency.

[ewbankkit]

Related:

• https://github.com/terraform-providers/terraform-provider-aws/issues/4418

[ionosphere80]

Still an issue in provider v3.2.0. The first application run usually taints the S3 bucket in one of the two regions. A second application run usually resolves the issue.