Terraform: Cant pass a list as a command line argument

[red8888]

Terraform Version

Terraform v0.11.1
+ provider.aws v1.6.0

Terraform Configuration Files

    // my tf file:
    variable "myvar" {type = "list"}

    module "my-module" {
      blah = "${var.myvar}"
      source = "path/to/module"
    }

various command line attempts (powershell)

    terraform plan -var myvar="zzzz"
    should be type list, got string

    terraform plan -var myvar=["zzzz"]
    invalid value "myvar=[zzzz]" for flag -var: Cannot parse value for variable ("[zzzz]") as valid HCL: At 1:6: unexpected token while parsing list: IDENT

    terraform plan -var 'myvar=["zzzz"]'
    invalid value "myvar=[zzzz]" for flag -var: Cannot parse value for variable ("[zzzz]") as valid HCL: At 1:6: unexpected token while parsing list: IDENT

[jbardin]

Hi @red8888,

Your last example is correct (and works) for most unix shells.
It appears that PowerShell is stripping the double quotes from the value before terraform can evaluate it. There's not much Terraform can do in this case, so we need to find a way to get the exact string to the process.

Does it work if you escape the double quotes with backticks? (though IIRC, PowerShell should preserve double quotes within single quote, so I'm not sure where the quotes are going in this case)

terraform plan -var 'myvar=[`"zzzz`"]'

[red8888]

nope:
invalid value "myvar=[zzzz]" for flag -var: Cannot parse value for variable ("[zzzz]") as valid HCL: At 1:6: illegal char

btw this doesn't work for cmd either.

PS has a literal escape thing that should work but this doesn't work either:
terraform --% plan -var 'myvar=["zzzz"]'
invalid value "'myvar=[zzzz]'" for flag -var: Cannot parse value for variable ("[zzzz]'") as valid HCL: At 1:6: unexpected token while parsing list: IDENT

for some reason it keeps putting the quotes on the outside of the brackets it seems

[red8888]

could this be a bug with the windows exe?

[jbardin]

Can you run the command with TF_LOG=trace? One of the first few lines will be the exact output of the CLI args as passed in by the OS, and then later as they are passed to the command function. This will at least verify whether terraform is seeing the quotes at all.

[red8888]

ok!

2018/01/03 19:35:40 [INFO] Terraform version: 0.11.1  a42fdb08a43c7fabb8898fe8c286b793bbaa4835+CHANGES
2018/01/03 19:35:40 [INFO] Go runtime version: go1.9
2018/01/03 19:35:40 [INFO] CLI args: []string{"c:\\terraform\\terraform.exe", "plan", "-var", "myvar=[zzzz]"}
2018/01/03 19:35:40 [DEBUG] Attempting to open CLI config file: C:\Users\meee\AppData\Roaming\terraform.rc
2018/01/03 19:35:40 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2018/01/03 19:35:40 [INFO] CLI command args: []string{"plan", "-var", "myvar=[zzzz]"}
invalid value "eb_environment2018/01/03 19:35:40 [DEBUG] plugin: waiting for all plugin processes to complete...
_name=[zzzz]" for flag -var: Cannot parse value for variable ("[zzzz]") as valid HCL: At 1:6: unexpected token while parsing list: IDENT
Usage: terraform plan [options] [DIR-OR-PLAN]

[red8888]

well this is horrid, but using that niffy trace seems like this actually works:
terraform plan -var 'eb_environment_name=["""env123"""]'

[red8888]

I _guess_ that solves it but I still think the exe itself is doing something funky. --% is a semi new feature that is supposed to pass a totally literal un-escaped string. Thats why I was thinking maybe the exe has some goofy logic baked in to deal with windows awfulness?

[jbardin]

Hi @red8888,

I'm sure there's some special escaping logic to deal with windows execution, but that is entirely outside Teraform. There might not be anything we can do about it, but I did just think of one last thing we can check to narrow down the scope.

Can you try setting TF_FORK=0 and seeing if that changes the value of the first CLI command args log line? It might also be worth verifying if the behavior is the same for cmd.exe.

Thanks!

[apparentlymart]

This reminds me of #11112, which was another issue where PowerShell was not parsing the command line in the way we expected.

Unfortunately when the Terraform team tests on Windows we tend to use cmd.exe rather than PowerShell, so we don't have a lot of experience with the details of its parser. It seems to be different enough from usual Unix shell parsing and cmd.exe-based parsing that unfortunately our documentation examples often don't work quite right with PowerShell.

When working on #11112 I referred to _Solve Problems with External Command Lines in PowerShell_, which includes some different ways to figure out how PowerShell is parsing a particular command line.

I think @red8888 is correct that there's some additional weirdness here due to the fact that command lines on Windows are not usually pre-tokenized by the shell (when PowerShell isn't in use) and so Go does some extra work on Windows to try to create a "Unix-like" os.Args array. It appears that when PowerShell is in use the command line is first tokenized by PowerShell, then serialized back into a string to launch the process, which Go then accesses using GetCommandLine before applying the logic linked earlier. Go apparently attempts to mimic the similar logic used by Microsoft's C runtime library as included in Visual Studio.

I found some useful guidance in _PowerShell and external commands done right_, which includes an explanation of why the quotes vanished in the original command line described here, and some techniques to mitigate it.

Unfortunately I don't have a lot of hope for a _great_ answer here, since we're at the mercy of PowerShell's logic for re-serializing the parsed command line. :confounded: For now, it's probably best to use -var-file instead of -var when passing complex values in PowerShell, though if we can get a good enough handle on what's going on here we could consider adding some PowerShell-specific information to the main _Commands_ documentation page to recommend some specific techniques for passing arguments to Terraform via PowerShell.

Perhaps there is some possibility to write a PowerShell cmdlet than can wrap Terraform and make it more PowerShell-friendly, but this isn't something we (the Terraform Team at HashiCorp) have the expertise to do, so unfortunately we'd need to delegate that to the community. :disappointed:

[thalesac]

I'm having a similar issue with Linux cli.
terraform plan -var 'myvar=["zzzz"]' indeed works, but the following doesn't:

• terraform plan -var 'myvar=['zzzz']'
• terraform plan -var "myvar=['zzzz']"
• terraform plan -var myvar=['zzzz']

Is there a reason for that?
At my current job we have a tool that generates the terraform/packer parameters out of custom YML files.
The python PYAML library get an entry like:

varlist: 
  - item1
  - item2

and transforms into ['item1', 'item2']
Can we reopen this ticket as a feature request to support that format?
As per of YAML specification and PYAML documentation it seems that this format is the expected.

What are your thoughts on that @apparentlymart ?

The only workaround I could come up is to create a temporarily tfvar file.
Thanks

[apparentlymart]

Hi @thalesac!

In Unix shells, quotes are interpreted by the shell to override the default behavior of splitting arguments with whitespace, and so the first "level" of quotes is consumed by the shell and thus isn't passed to Terraform at all. The first example works because the ' quotes prevent special interpretation of the inner " quotes and so they are passed by the shell into the command line Terraform sees.

Unfortunately all of these behaviors are built into the shell and Terraform cannot do anything to disable or customize them. Placing the whole argument in single quotes as in your first example is the most robust method.

(Note that Terraform doesn't support single quotes as string delimiters in its expression language, so the quotes passes literally to Terraform must always be double quotes.)

[Ali-TW-London]

I used this ways and works well with latest version of Terraform
terraform plan -var 'availability_zones=["one","two","three"]'

[Sudharma]

this still does not work for me.. any hints?

[borissnd]

I had a similar issue when running terraform 0.11 on my Mac.
terraform would not accept spaces between elements in the list.
this works:
-var 'day=["sunday","monday"]'
this does not:
-var 'day=["sunday", "monday"]'

[TechnicallyJoe]

well this is horrid, but using that niffy trace seems like this actually works:
terraform plan -var 'eb_environment_name=["""env123"""]'

Well, that worked for me. Windows 10, version 2004 - PS version 7.0.3

Ugly, but it works..

[kubealex]

Still not working in 0.14.3, no way to pass a list of strings from command line.

[root@heisenberg libvirt-resources]# terraform apply -var 'network_cidr=["10.1.1.1/24"]'

Error: Incorrect attribute value type

  on libvirt-resources.tf line 33, in resource "libvirt_network" "kube_network":
  33:   addresses = var.network_cidr
    |----------------
    | var.network_cidr is "[\"10.1.1.1/24\"]"

Inappropriate value for attribute "addresses": list of string required.

[root@heisenberg libvirt-resources]# terraform apply -var 'network_cidr=["lol", "in", "california"]'

Error: Incorrect attribute value type

  on libvirt-resources.tf line 33, in resource "libvirt_network" "kube_network":
  33:   addresses = var.network_cidr
    |----------------
    | var.network_cidr is "[\"lol\", \"in\", \"california\"]"

Inappropriate value for attribute "addresses": list of string required.