Terragrunt: Need to be able to specify which AWS profile to use

[Japh]

I've just setup Terragrunt, and after running a couple of times went to check DynamoDB to see what's happening there. Every time I've run it, the output has included something like the following:

Attempting to acquire lock for state file terraform-example in DynamoDB
Attempting to create lock item for state file terraform-example in DynamoDB table terragrunt_locks
Lock acquired!

Yet, I can't locate any such table in AWS. I've checked all regions. Originally, I wasn't specifying a region, which I've updated now. But still, no actual table.

So my question is... is the lock really happening? Or is Terragrunt failing silently in some way and giving me a false sense of security?

(I'm not running Terragrunt on an EC2 instance here, I'm running it on my local machine. Which I assume is normal, but something I read elsewhere made me wonder about this.)

[Japh]

I've done some further testing, and the lock _does_ appear to be in effect. But I still cannot locate the DynamoDB table being used.

I do have multiple AWS profiles going, so perhaps it's using the default profile rather than the one I'm using for the state storage?

[Japh]

If that is the case, how do I specify the AWS profile to use in the locking config? (Feel free to rename this issue to more accurately reflect that it's _probably_ really a request for the ability to specify a profile.)

[Japh]

Can confirm... it's using DynamoDB on my other AWS profile.

[josh-padnick]

@Japh Thanks for your note. A couple of thoughts:

First, if you did want to "persist" the DynamoDB Lock, you could always start a terragrunt run and then terminate it as soon as the lock is acquired with CTRL-C. Then you have all the time you need to find it in DynamoDB. When you do find it in DynamoDB, you can manually delete the entry to remove the lock, or use terragrunt release-lock to do it.

Second, all of terragrunt's AWS authentication logic comes from AWS SDK for Go, which is documented here. On that page, you'll see that if you want to select a non-default profile, you need to set the environment variable AWS_PROFILE to the name of the profile.

Does that address your question?

[Japh]

@josh-padnick Thanks for your reply!

The second part there does _kind of_ answer my question. But given that I can specify a profile to use in my main.tf, it'd be nice if Terragrunt respected that.

My state files are being stored in an S3 bucket for the profile specified in main.tf, but Terragrunt is creating and using a DynamoDB table in my default profile instead. That's not expected behaviour.

Given that there is a mechanism, as you point out, to use a different profile, Terragrunt should allow for that and handle it, I would think. Right?

[brikis98]

@Japh Having Terragrunt figure out the AWS profile to use from your code is a fairly complicated endeavor: we'd have to read all your Terraform files, parse not just HCL syntax, but Terraform's particular interpretation of it, find all the providers (there may be more than one), dig through all your modules (which could also have providers), process variables and interpolations, and a whole bunch of other non-trivial logic. It would not only be hard to build, but also brittle, as Terraform could change some of this logic, and then we'd be playing a catch-up game to implement it.

As explained in #21, the best way to handle all auth details with Terragrunt is to use the typical mechanisms supported by all AWS apps: that is, environment variables such as AWS_ACCESS_KEY and AWS_PROFILE, IAM roles, or logging in via the AWS CLI, which puts auth details in ~/.aws. I don't think it's worth supporting other mechanisms at this point.

[Japh]

@brikis98 No worries, appreciate your reply. I was thinking it could be as simple as supporting a parameter in the .terragrunt configuration, much the same as you can do with the provider in a main.tf. But I'll stick with specifying an environment variable instead.

Thanks!

[josh-padnick]

@Japh Just to clarify, your AWS profile configuration is inherently specific to your local environment. Other developers will have different local profile configurations. That's another reason why it wouldn't make sense to persist a profile name in a .terragrunt file, but setting an environment variable does make sense.

[Japh]

@josh-padnick Ah of course! Excellent point. In fact... I should re-think using it in my main.tf for this reason also... Thanks! 😄