Terragrunt: What is the best way to manage access to the Terraform remote backend (like S3) ?

[barryib]

Actually Terragrunt can create remote backend on S3 and GCP with audit, encryption activated. But how can we manage access to that backend automatically ?

Does Terragrunt create a bucket policy ? If not, what is the best way to manage this ?

[yorinasub17]

Terragrunt currently doesn't do anything related to managing permissions on the bucket. This is a bit of an advanced configuration for terragrunt to handle as a part of automatically creating buckets. For something like this, it is probably better off managed in terraform.

See https://github.com/gruntwork-io/terragrunt/issues/859#issuecomment-530321902 for a workflow to accomplish this.

[barryib]

I mostly use Terragrunt for its DRY implementation and its remote backend creation. The latest eliminate a lot of complexity in the bucket initialisation workflow. I would love to continue using that feature.

Actually Terragrunt try to enforce a sort of best practices for remote backend configuration like encryption, versioning, audit, block public acess, etc. But there is something missing. In fact, in remote states, you can have secrets and passwords, so we need something to deny/allow access from unauthorized/authorized roles.

I understand your point. But is applying bucket policy add a lot of complexity ? The idea here is to provide bucket policy (in json for exemple) to Terragrunt and let him apply it. The AWS SDK provide already method for that.

I can quickly open a PR for this.

@brikis98 @yorinasub17 are you open to discussion or this is a NO GO for you ?

[yorinasub17]

The challenge is encoding this in a way in terragrunt.hcl such that it is not too confusing. For example, right now we have mixed terragrunt specific configurations of the S3 bucket with terraform config for the remote state backend. This leads to some confusions about which configuration maps to what.

Also, if we trend towards this direction, we are basically committing to implementing the full S3 bucket API for managing it, which duplicates logic available with terraform.

Thinking out loud, the ideal solution I would be looking for for a feature like this would be to switch the terragrunt bucket creation routine to support the terraform API. It would be really nice if you could write out the terraform resource block for S3 buckets and DynamoDB tables somewhere (possibly embedded as a subblock in remote_state?), and then terragrunt uses that (with the AWS provider in the backend) to create the state backend resources. It would be even better if we could store the state somewhere so that you could destroy the resources in the end.

[yorinasub17]

In either case, we are open to discussion for enhancing the bucket creation routine, but need to see some designs on how it would be encoded and various possible directions. We are unlikely to accept a PR that simply tacks on yet another config for the bucket creation API, but if there is some clever way you can come up with to make it easier to extend the API beyond just the bucket policy that would be great!

Given that such a feature would fundamentally change the bucket creation workflow, the first step for proposing such a change would be to write an RFC along the veins of https://github.com/gruntwork-io/terragrunt/blob/master/_docs/rfc/for_each_iteration.md and https://github.com/gruntwork-io/terragrunt/pull/882