Terraform: Unexpected behaviour with `terraform_remote_state` `config` block

[HemantNegi]

Terraform Version

Terraform v0.11.13
+ provider.aws v1.60.0
+ provider.kubernetes v1.5.2
+ provider.vault v1.7.0

Terraform Configuration Files

I am using terraform_remote_state to fetch a .tfstate from s3. I am seeing a weird behaviour.
When I run terraform validate or terraform plan. Below piece of config works as expected.

data "terraform_remote_state" "external_ingress" {
  backend = "s3"

  config  = {
    bucket         = "my-project-terraform-state"
    encrypt        = "true"
    region         = "us-west-2"
    dynamodb_table = "my-project-terraform-locks"
    kms_key_id     = "arn:aws:kms:us-west-2:056324545554:alias/my-project-state-key"
    key            = "myproject/terraform/external-ingress/terraform.tfstate"
  }
}

But this gives out an error:

variable "ecosystem_config" {
  type = "map"
  default = {
    bucket         = "my-project-terraform-state"
    encrypt        = "true"
    region         = "us-west-2"
    dynamodb_table = "my-project-terraform-locks"
    kms_key_id     = "arn:aws:kms:us-west-2:056324545554:alias/my-project-state-key"
  }
}

data "terraform_remote_state" "external_ingress" {
  backend = "s3"

  config = "${merge(
    var.ecosystem_config,
    map("key", "myproject/terraform/external-ingress/terraform.tfstate")
  )}"
}

Debug Output

.
.
data.terraform_remote_state.external_ingress - *terraform.NodeValidatableResourceInstance
2019/04/10 16:21:44 [ERROR] root: eval: *terraform.EvalValidateResource, err: Warnings: []. Errors: [config: should be a list]
2019/04/10 16:21:44 [ERROR] root: eval: *terraform.EvalSequence, err: Warnings: []. Errors: [config: should be a list]

2019/04/10 16:21:44 [DEBUG] plugin: waiting for all plugin processes to complete...
Error: data.terraform_remote_state.external_ingress: config: should be a list

Crash Output

Error: data.terraform_remote_state.external_ingress: config: should be a list

Expected Behavior

It should have run normally as the first part did.

Actual Behavior

It throws out this error and plan/validate/apply phase is not completed.

Steps to Reproduce

on the second config run following:

terraform validate

terraform plan

It seems like a problem with interpolation syntax, so posted this issue here. Thanks.

[jbardin]

Hi @HemantNegi

Sorry about the confusion here. As you mention, config is a "block"., which does not use = for assignment. It works in the first case because of some flexibility in the underlying config language, but that is going to be better enforced in the next release.

What this means is that you should not be assigning a map value to a block, because they are essentially different things. While there are same ways to get it to work in 0.11, like assigning it within a list, I would recommend setting the individual attributes for forward compatibility and to prevent possible unexpected results.

[jbardin]

Hi @HemantNegi,

Sorry, my last answer wasn't quite correct (but I'll leave it for reference), because it turns out we've marked config as a map _attribute_ in 0.12, which means what you are trying to do will work, but only in 0.12. Any use of the config attribute as a block in 0.11 will be fixed up by the upgrade tool to add = when appropriate.

This still leaves mostly the same result here however, and if you want to interpolate the config values it will be best to set them individually for now.

[HemantNegi]

Hi @jbardin,

Thanks for your reply.
I got your point that config is marked as a map attribute from version 0.12. I tried running it with version 0.11.8 and it worked. My question here is if interpolation syntax with config (or map type attributes) is no longer supported then what are the alternatives we have, to use a partial configuration here. I don't want to put all the config values (ecosystem_config in my case) in a single block as I have to repeat them all over the place which will be hard to maintain.

[apparentlymart]

Hi @HemantNegi,

What you tried in your original example will work in Terraform v0.12... that is part of the reason why it is becoming an attribute in the first place.

You may be able to work around it in Terraform v0.11 by further exploiting the missing validation that caused that confusing error in the first place: you could try putting list brackets around the expression to force it to be treated as a list with a single not-yet-known object element rather than a not-yet-known list:

  config = ["${merge(
    var.ecosystem_config,
    map("key", "myproject/terraform/external-ingress/terraform.tfstate")
  )}"]

I'm not sure if the above will work (it's exploiting a bug to work around that same bug), and those brackets will definitely need to be removed again after upgrading to Terraform v0.12, but it may help you get what you need in the meantime, without enumerating all of the S3 bucket arguments individually.

[HemantNegi]

Hi @apparentlymart,

Wrapping the interpolation expression in a list worked with v0.11.13
This will work for me now, Will look forward to v0.12
Thanks for your help.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.