Terraform: Enhancement: Configure backend through environment variables similar to tfvars

[eedwards-sk]

Current Terraform Version

0.11.10

Use-cases

In a CI pipeline, configuring the terraform commands through environment variables rather than explicit arguments is a highly useful feature.

We currently can provide terraform variable values from the environment by using the TF_VAR_ prefix.

However, we currently cannot use environment variables to configure either the backend type or the backend config values for the init command.

Attempted Solutions

At the moment, the workaround is to use another tool externally to process environment variables similar to as proposed, and then provide them as explicit args.

getting values from environment

# =============================================================================
# _get_backend_type_from_environment
# =============================================================================
def _get_backend_type_from_environment() -> Optional[str]:
    return os.environ.get(BACKEND_TYPE_VAR)


# =============================================================================
# _get_backend_config_from_environment
# =============================================================================
def _get_backend_config_from_environment() -> Optional[dict]:
    backend_config: dict = {}
    for key, value in os.environ.items():
        if key.startswith(BACKEND_CONFIG_VAR_PREFIX):
            # strip prefix and use the remainder as the key name
            backend_config[key[len(BACKEND_CONFIG_VAR_PREFIX):]] = value
    return backend_config or None

converting them to explicit args

    # set backend config values
    if backend_config_vars:
        for k, v in backend_config_vars.items():
            terraform_command_args.append(
                f"-backend-config=\"{k}={v}\"")

Proposal

I propose we add two new environment variables that can be used to configure the backend during init:

backend type: TF_BACKEND_TYPE

e.g. instead of

terraform {
  backend "s3" {
  }
}

one could provide

TF_BACKEND_TYPE=s3 terraform init

backend config: TF_BACKEND_CONFIG_<var>

e.g. instead of

terraform init \
  -backend-config="bucket=my-bucket"

one could provide

TF_BACKEND_CONFIG_bucket="my-bucket" terraform init

[apparentlymart]

Hi @eedwards-sk! Thanks for sharing this use-case.

This seems like a reasonable idea to me.

One detail I'd like to think some more about is how to helper users understand that these environment variables apply only during terraform init and that they can't just reset them between other commands. Right now Terraform explicitly tests to make sure the backend configuration hasn't changed since the last terraform init so it can block running other commands until the backend has been re-initialized, to avoid misleading users into thinking they are working against a different configuration and then doing something potentially-destructive like writing changes to the wrong state file.

When we introduced the TF_WORKSPACE environment variable (for a similar use-case) the compromise we settled on was that terraform workspace select would produce an error if asked to change workspace while the environment variable is set, since otherwise you might be misled into thinking you changed workspace even though the environment variable is overriding it.

I think we'd need to take a similar approach here: during any command that works with the backend, we'd need to check whether these environment variables are still set, incorporate their effect into the resolved backend configuration, and produce an error if anything has changed since the last terraform init, requiring the user to run terraform init explicitly again to confirm the intended change.

---

With this said, I wonder if the existing TF_CLI_ARGS_ environment variables mechanism is good enough here. This mechanism allows setting any command line argument for a particular command via the environment, like this:

$ export TF_CLI_ARGS_init='-backend=s3 -backend-config="..."'
$ terraform init

Terraform will then treat these as additional arguments whenever terraform init is run. This doesn't work for all situations due to where Terraform inserts these additional arguments into the list of those given on the command line, but I think it would work here where there are no other arguments anyway.

This mechanism was added specifically to avoid the need to make distinct environment variable analogs to every command line option, and the naming schema TF_CLI_ARGS_init makes it explicit (hopefully) that this applies only to the terraform init command, avoiding my concern above.

If this approach is functional then I think we'd prefer to stick with this since this gives fewer distinct permutations to document, understand, test, etc.

[eedwards-sk]

One detail I'd like to think some more about is how to helper users understand that these environment variables apply only during terraform init and that they can't just reset them between other commands.

That's a very valid concern. As someone who has mostly used terraform before backends, workspaces, or anything fancy involving state, the init process was new to me, and I quickly learned that backend configuration is an 'init only' process.

I think the behavior became clear to me quickly because I'm working in a ci-pipeline situation where I go from init to a 'plan artifact', so the init wouldn't make sense again anyway since my 'apply' phase requires a plan artifact. For users using an interactive / manual workflow, their onboarding process might be different.

With this said, I wonder if the existing TF_CLI_ARGS_ environment variables mechanism is good enough here.

Did not know about that one. My only concern is that it becomes a lot uglier of an interface for a user in my use case.

e.g. my configuration from a concourse task is limited to environment variables, so passing backend config down to terraform through concourse -> python -> terraform with my above helper code is as easy as:

- name: terraform-plan
  plan:
  - get: source
  - get: concourse-terraform
  - get: concourse-terraform-image
  - task: terraform-plan
    image: concourse-terraform-image
    file: concourse-terraform/tasks/plan.yaml
    input_mapping:
      terraform-source-dir: source
    params:
      TF_WORKING_DIR: source/terraform
      TF_BACKEND_TYPE: s3
      TF_BACKEND_CONFIG_bucket: ((aws_bucket))
      TF_BACKEND_CONFIG_key: ((aws_bucket_key))
      TF_BACKEND_CONFIG_region: ((aws_region))

If I understand TF_CLI_ARGS_init correctly, it would instead look like this:

- name: terraform-plan
  plan:
  - get: source
  - get: concourse-terraform
  - get: concourse-terraform-image
  - task: terraform-plan
    image: concourse-terraform-image
    file: concourse-terraform/tasks/plan.yaml
    input_mapping:
      terraform-source-dir: source
    params:
      TF_WORKING_DIR: source/terraform
      TF_BACKEND_TYPE: s3
      TF_CLI_ARGS_init: '-backend-config="bucket=((aws_bucket))" -backend-config="key=((aws_bucket_key))" -backend-config="region=((aws_region))"'

So it looks doable, but I feel like there's a lot more room to mess up the configuration since now the end-user has to understand the command line rather than just the parameters.

In addition, this doesn't seem to solve the situation of needing to generate a config file to specify the backend type. e.g. I still have to turn TF_BACKEND_TYPE: s3 into a backend.tf file with backend "s3" {}, so having that as at least a CLI option that I can then set with TF_CLI_ARGS_init would be nice.

Although... actually... in your example... -backend=s3 ...? does that work?

If so it's undocumented, but good to know :)

Edit: -backend seems to only support a boolean value

→ terraform init -backend=local
invalid boolean value "local" for -backend: strconv.ParseBool: parsing "local": invalid syntax

Thanks for your response.

[apparentlymart]

Hi @eedwards-sk! Thanks for all that feedback.

You're indeed right that setting the backend on the command line like that isn't supported. I was too hasty in suggesting that.

I believe it was intentional that the backend type can be selected only by configuration and not overridden on the command line, but off the top of my head I don't know why that was (it was before I joined the team) so we'll need to figure that out as part of considering this, to see if that reason is still valid since I expect that reason would also apply to setting it via an environment variable too. The team is currently very focused on addressing the remaining work for the forthcoming v0.12.0 release so I won't be able to dig into that right now, but we'll revisit this as part of future work to revisit the backend configuration mechanisms in general. (I've labelled this additionally with "backend" to remind us to include this in that effort.)

Thanks again for sharing that additional context!

[osterman]

@eedwards-sk we added support to pass 100% of terraform arguments via environment variables. The suggestion by @apparentlymart to use TF_CLI_ARGS* was brilliant. I hadn't thought to use them like this, but it works nicely.

So in your example, you want to do this:

terraform init \
  -backend-config="bucket=my-bucket"

With tfenv, you can do this in your shell (or .envrc with direnv):

export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=my-bucket
source <(tfenv)
terraform init

Or as a thin wrapper...

export TF_CLI_INIT_BACKEND_CONFIG_BUCKET=my-bucket
tfenv terraform init

Read more here: https://github.com/cloudposse/tfenv

[vojtechmares]

Hello, I would like to know if this has been implemented or not. Apperently this issue has been open for about 2 years and I couldn't find any information about this in documentation.

Is it implemented or not?

In GitLab there is a merge request counting on using these environment variables. It is targeting version 13.2 with feature freeze on 17th July.