Terragrunt: Dealing with Hard Coded Credentials for Private Repos

[tomstockton]

Hi Terragrunt team!

First of all, love your work, I've bought the 'Up & Running' book and am following your advice on using Terraform as closely as possible!

Specifically, I'm using the pipelines pattern that you referenced and have created the following structure:

````
project_live repo:
environment/
component/
terraform.tfvars # defines git source to private 'project_modules repo'

project_modules repo:
component/
main.tf # defines modules from the private 'core_modules repo'

code_modules repo:
module/
main.tf # defines groups of resources
````

All these repos are private, hence needing to define in the format (as per the documentation) which hard codes user credentials into the git URL. Obviously this isn't ideal as has been recognised in https://github.com/hashicorp/terraform/issues/1439 and I'm looking for a better solution!

Do you guys have any thoughts on how to deal with this? Would it be a worthwhile feature that could be added to Terragrunt in order to allow variable interpolation into the source parameter?

I've achieved this with some success using a wrapper shell script to replace string {VCS_CREDS} with env var ${VCS_CREDS} but it only works as far as pulling the project_modules repo. Any references in project_modules to core_modules still have to be hardcoded due to terraform init <url> downloading any modules defined in project_modules.

Thanks!

[brikis98]

Use SSH URLs.

terragrunt = {
  terraform {
    source = "git::[email protected]:foo/bar.git//baz?ref=v1.0.8"
  }
}

This will use your SSH creds, so no need to store any passwords in the code itself.

[tomstockton]

Ah, thanks. I'd tried this but it failed. I'm using bitbucket rather than github though and it seems that Terraform interprets git::[email protected]:foo/bar.git//baz as a reference to a local path.

I'll create an issue / PR for terraform to properly handle bitbucket SSH URLs.

[brikis98]

For BitBucket, try:

git::ssh://[email protected]:foo/bar.git//baz

[tomstockton]

Almost! Running that gives:

ssh: Could not resolve hostname bitbucket.org:foo: nodename nor servname provided, or not known

However if I change it to:

git::ssh://[email protected]/foo/bar.git//baz

Then it works! Not sure if this is intended usage but it's a nice workaround in any case.

Thanks again.

[brikis98]

Nice! Thanks for following up.

[fosrias]

I really think it would be helpful to add an example of this to the terragrunt documentation because to get this to actually work, combined with this less than helpful error message, https://github.com/gruntwork-io/terragrunt/issues/163, takes a fair amount of digging around.

[brikis98]

@fosrias Agreed! PRs are very welcome :)

[fosrias]

@brikis98 K. Will do. However, I ran into one issue using Terragrunt in a jenkins pipeline. It was necessary to run the following step to to auto accept github as a host prior to using Terragrunt: ssh -T -oStrictHostKeyChecking=no [email protected] || true.

If not, then you would get an error of:

Host key verification failed.
fatal: Could not read from remote repository.

I can add this to the documentation, but part of me wonders if Terragrunt needs to either accept -oStrictHostKeyChecking=no or have its own switch. Thoughts?

[brikis98]

@fosrias The git clone and all associated permissions are handled 100% by Terraform, not Terragrunt, so I'm hesitant to add any ssh-specific behavior to Terragrunt itself. However, I'm totally in favor of adding this to the README.