Atlantis: Using tfenv to manage multiple Terraform binaries?

[stephenchu]

Hi.

Just wondering if there is any appetite in using tfenv to manage multiple versions of Terraform binaries, such that Git repos can use their own version of Terraform (via a .terraform-version) file, without having to rely on whatever the atlantis Dockerfile supports?

If the direction is set, probably can contribute a PR here.

[lkysow]

Hi Stephen,
Atlantis already supports multiple versions via the repo-level atlantis.yaml file (see https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#terraform-versions):

version: 3
projects:
- dir: .
  terraform_version: v0.11.12

Atlantis will automatically download the version specified, you don't need to include it in the Docker image anymore.

Given the above, do you think tfenv is still needed?

[cyrus-mc]

@lkysow I am attempting to use the terraform_version setting and I find it isn't working.

version: 3
projects:
- name: test
  dir: .
  workflow: version
  terraform_version: v0.11.14
workflows:
  version:
    plan:
      steps:
        - run: terraform -v

As you can see the plan simply outputs the version of terraform. When I run this it outputs 0.11.13 which is the version of 0.11 included in the Atlantis image. My main use case is to use Terraform 0.12.0-rc1 (which I actually download and built my own image).

[lkysow]

Hey, custom run steps are just run as-is so they will use whatever binary is called terraform in the path. If you weren't using custom workflows it would use the correct version of Terraform.

Really we should offer this as an env var, but for now you can use ~/.atlantis/bin/terraform0.11.14:

        - run: ~/.atlantis/bin/terraform0.11.14 -v

[cyrus-mc]

@lkysow

That is what I thought. Better would be to just symlink terraform to terraform_version for that specific workflow.

[lkysow]

I'm going to close this because I don't think it makes sense given the existing Terraform version support.

[smiller171]

@lkysow I think tfenv support would still be really nice so that I don't have to remember to update atlantis.yaml when I update .terraform-version or the terraform required_version config. Additionally tfenv can automatically select the correct version from a required_version config whereas the current versioning logic can't.

If you were to replace the existing version management logic with tfenv and pass theatlantis.yaml terraform_version to tfenv instead of what you're doing currently I think it could be a significant improvement. Alternately simply supporting looking for a .terraform-version file in the existing version management system would be great as well.

[LBates2000]

I was able to set up Atlantis to use both tfenv and tgenv so that I can use terragrunt with both terraform 0.11.x and 0.12.x. Because I am using terragrunt, I had already enabled custom workflows, which I'm currently using with an atlantis.yaml file in the root of a monolithic repo. I chose to update the atlantis.yaml instead of using individual .terraform_version and .terrgrunt_version files so as to preserve the existing workflow. The relevent bits are below.

Installing tfenv and tgenv in my Atlantis install script. This is done immediately before installing the Atlantis binary and configuring it as a service:

terraform_version='0.11.8'
terragrunt_version='0.18.7'

# Install tfenv and tgenv, and use to install default terraform and terragrunt versions
git clone https://github.com/tfutils/tfenv.git ~/.tfenv
ln -s ~/.tfenv/bin/* /usr/local/bin
git clone https://github.com/cunymatthieu/tgenv.git ~/.tgenv
ln -s ~/.tgenv/bin/* /usr/local/bin
. ${HOME}.profile
tfenv install $terraform_version
tfenv use $terraform_version
tgenv install $terragrunt_version
tgenv use $terragrunt_version

Example of projects and workflows for terraform 0.11.x and 0.12.x in atlantis.yaml:

- dir: infrastructure/terraform11/projects/aws/stage/us-east-1
  workspace: default
  terraform_version: 0.11.8
  autoplan:
    when_modified: ["*.tf", "../modules/**.tf", "*.tfplan"]
    enabled: false
  workflow: terragrunt
- dir: infrastructure/terraform12/projects/aws/stage/us-east-1
  workspace: default
  terraform_version: 0.12.20
  autoplan:
    when_modified: ["*.tf", "../modules/**.tf", "*.tfplan"]
    enabled: false
  workflow: terragrunt12
workflows:
  terragrunt:     
    plan:
      steps:
      - env:    
          name: ATLANTIS_TERRAGRUNT_VERSION
          value: 0.18.7 
      - run: tfenv install ${ATLANTIS_TERRAFORM_VERSION}
      - run: tfenv use ${ATLANTIS_TERRAFORM_VERSION}
      - run: tgenv install ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: tgenv use ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: terragrunt init -no-color
      - run: terragrunt plan -no-color -out=$PLANFILE | landscape --no-color
    apply:
      steps:
      - env:    
          name: ATLANTIS_TERRAGRUNT_VERSION
          value: 0.18.7     
      - run: tfenv install ${ATLANTIS_TERRAFORM_VERSION}
      - run: tfenv use ${ATLANTIS_TERRAFORM_VERSION}
      - run: tgenv install ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: tgenv use ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: terragrunt apply -no-color $PLANFILE
  terragrunt12:      
    plan:
      steps:      
      - env:    
          name: ATLANTIS_TERRAGRUNT_VERSION
          value: 0.21.11          
      - run: tfenv install ${ATLANTIS_TERRAFORM_VERSION}
      - run: tfenv use ${ATLANTIS_TERRAFORM_VERSION}
      - run: tgenv install ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: tgenv use ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: terragrunt init -no-color
      - run: terragrunt plan -no-color -out=$PLANFILE | landscape --no-color
    apply:
      steps:
      - env:    
          name: ATLANTIS_TERRAGRUNT_VERSION
          value: 0.21.11
      - run: tfenv install ${ATLANTIS_TERRAFORM_VERSION}
      - run: tfenv use ${ATLANTIS_TERRAFORM_VERSION}
      - run: tgenv install ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: tgenv use ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: terragrunt apply -no-color $PLANFILE

The redundancy in the plan and apply stages was a safeguard against other PRs being worked on between the time a PR has been planned and when it is applied. In that time, another PR could change the terraform and terragrunt versions on the Atlantis server. Setting the versions in each stage ensures the right versions are always used at plan and apply time.

[lkysow]

@LBates2000 that's awesome, thanks for sharing!

[LBates2000]

More concise version that uses a single workflow for both versions:

- dir: infrastructure/terraform11/projects/aws/stage/us-east-1
  workspace: default
  terraform_version: 0.11.8
  autoplan:
    when_modified: ["*.tf", "../modules/**.tf", "*.tfplan"]
    enabled: false
  workflow: terragrunt
- dir: infrastructure/terraform12/projects/aws/stage/us-east-1
  workspace: default
  terraform_version: 0.12.20
  autoplan:
    when_modified: ["*.tf", "../modules/**.tf", "*.tfplan"]
    enabled: false
  workflow: terragrunt
workflows:
  terragrunt:
    plan:
      steps:
      - env:
          name: ATLANTIS_TERRAGRUNT_VERSION
          command: echo $(if test `echo ${ATLANTIS_TERRAFORM_VERSION} | cut -d "." -f2` -eq '12' ; then echo "0.21.11" ; else echo "0.18.7" ; fi)
      - run: tfenv install ${ATLANTIS_TERRAFORM_VERSION}
      - run: tfenv use ${ATLANTIS_TERRAFORM_VERSION}
      - run: tgenv install ${ATLANTIS_TERRAGRUNT_VERSION} 
      - run: tgenv use ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: terragrunt init -no-color
      - run: terragrunt plan -no-color -out=$PLANFILE | landscape --no-color
    apply:
      steps:
      - run: tfenv install ${ATLANTIS_TERRAFORM_VERSION}
      - run: tfenv use ${ATLANTIS_TERRAFORM_VERSION}
      - run: tgenv install ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: tgenv use ${ATLANTIS_TERRAGRUNT_VERSION}
      - run: terragrunt apply -no-color $PLANFILE

[nitrocode]

Instead of specifying the version in the Atlantis yaml file, Id much rather either

1. use a .terraform-version file directly in the terraform module and have Atlantis pick that up and choose the correct module. All our modules already have this file and if they don't, they can use the default certain that Atlantis uses.

2. Even better would be detecting the last used terraform version in the state and using that version.

Thoughts?

[lkysow]

Instead of specifying the version in the Atlantis yaml file, Id much rather either

1. use a .terraform-version file directly in the terraform module and have Atlantis pick that up and choose the correct module. All our modules already have this file and if they don't, they can use the default certain that Atlantis uses.
2. Even better would be detecting the last used terraform version in the state and using that version.

Thoughts?

We detect if you've set

terraform {
  required_version
}

Is there a reason to use .terraform-version instead?

[nitrocode]

Oh i didn't know this! We use the .terraform-version so we can quickly switch to the correct version using tfenv locally.

[smiller171]

Instead of specifying the version in the Atlantis yaml file, Id much rather either

1. use a .terraform-version file directly in the terraform module and have Atlantis pick that up and choose the correct module. All our modules already have this file and if they don't, they can use the default certain that Atlantis uses.
2. Even better would be detecting the last used terraform version in the state and using that version.

Thoughts?

We detect if you've set

terraform {
  required_version
}

Is there a reason to use .terraform-version instead?

You only kindof detect required_version though. You don't properly parse the version string, you instead take an extremely minimal approach that limits the version strings that can be used.

[nitrocode]

We're using 0.14.0 and have this in our versions.tf

terraform {
  required_version = ">= 0.12.28"
}

Also tried this

terraform {
  required_version = "= 0.12.28"
}

And see this error when running the command/comment atlantis plan -d iam-policies/sre/

Error: Unsupported Terraform Core version

This configuration does not support Terraform version 0.12.27. To proceed,
either choose another supported Terraform version or update the root module's
version constraint. Version constraints are normally set for good reason, so
updating the constraint may lead to other errors or unexpected behavior.

Is this a bug/regression with the current version of atlantis?

[lkysow]

@nitrocode I've opened up https://github.com/runatlantis/atlantis/issues/1146 to discuss.

[Symbianx]

We're using tfenv as well to install and use the correct terraform version for each project by setting a custom workflow in atlantis.
However, this approach this fails when multiple plans are ran at the same time with different version requirements. This causes a race condition when 2 projects select different versions at the same time, causing one of the plans to fail because of a version mismatch.

@lkysow is there someway we can use atlantis' version management with custom workflow run steps?