Terraform: Unable to load variables using terraform.tfvars - AzureRM

[leowmjw]

Using the basic example from the AzureRM provider docs; it does not seem to be loading variables via file and does not work unless through Environment Variable.

As far as I can see; I am following the correct syntax; new user :) Hope can give guidance; I'm sure it is likely a silly beginners mistake.

Terraform Version

Terraform v0.7.0-dev (d85007b55a3b7553b7f855241a392e0596895144); also observed in v0.6.16

Affected Resource(s)

• azurerm Provider
  

  Terraform Configuration Files

variable "ARM_SUBSCRIPTION_ID"
{
  description = "ARM SUBSCRIPTION"
}

variable "azure_access_key" {
  description = "Access key for Azure"
}

variable "azure_secret_key" {
  description = "Secret Key for Azure"
}

variable "azure_subscription_id" {
  description = "Subscription ID; get from Console .."
}

variable "azure_tenant_id" {
  description = "Tenant ID; from EndPoint in classic panel .."
}

variable "azure_region" {
  description = "Region where we will operate; default to SG"
  default = "Southeast Asia"
}

provider "azurerm" {
  subscription_id = "${var.ARM_SUBSCRIPTION_ID}"
  client_id = "${var.azure_access_key}"
  client_secret = "${var.azure_secret_key}"
  tenant_id = "${var.azure_tenant_id}"
  alias = "sinar_proj"
}

# Create a resource group

resource "azurerm_resource_group" "development" {
  name = "development"
  location = "${var.azure_region}"
  tags {
    environment = "Development"
  }
}


# Create a virtual network in the web_servers resource group

resource "azurerm_virtual_network" "network" {
  name = "developmentNetwork"
  address_space = [
    "10.0.0.0/16"]
  location = "${var.azure_region}"
  resource_group_name = "${azurerm_resource_group.development.name}"

  subnet {
    name = "subnet1"
    address_prefix = "10.0.1.0/24"
  }

  subnet {
    name = "subnet2"
    address_prefix = "10.0.2.0/24"
  }

  subnet {
    name = "subnet3"
    address_prefix = "10.0.3.0/24"
  }
}

Debug Output

https://gist.github.com/leowmjw/692d899da18ab80e78b4ff4cba3115af

Expected Behavior

Load variables from terraform.tfvars

Actual Behavior

Does not seem to load from terraform.tfvars

Steps to Reproduce

Fails:

1. terraform plan

Error refreshing state: 1 error(s) occurred:

* 4 error(s) occurred:

* Subscription ID must be configured for the AzureRM provider
* Client ID must be configured for the AzureRM provider
* Client Secret must be configured for the AzureRM provider
* Tenant ID must be configured for the AzureRM provider

Succeeds:

1. ARM_SUBSCRIPTION_ID=abcd ARM_CLIENT_ID=efgh ARM_CLIENT_SECRET=ijkl ARM_TENANT_ID=xyz terraform plan

+ azurerm_resource_group.development
    location:         "southeastasia"
    name:             "development"
    tags.%:           "1"
    tags.environment: "Development"

+ azurerm_virtual_network.network
    address_space.#:                  "1"
    address_space.0:                  "10.0.0.0/16"
    location:                         "southeastasia"
    name:                             "developmentNetwork"
    resource_group_name:              "development"
    subnet.#:                         "3"
    subnet.1472110187.address_prefix: "10.0.1.0/24"
    subnet.1472110187.name:           "subnet1"
    subnet.1472110187.security_group: ""
    subnet.2796830261.address_prefix: "10.0.2.0/24"
    subnet.2796830261.name:           "subnet2"
    subnet.2796830261.security_group: ""
    subnet.4132282879.address_prefix: "10.0.3.0/24"
    subnet.4132282879.name:           "subnet3"
    subnet.4132282879.security_group: ""


Plan: 2 to add, 0 to change, 0 to destroy.

Important Factoids

leow$ cat terraform.tfvars
ARM_SUBSCRIPTION_ID = "efgh"
azure_access_key = "xxx"
azure_secret_key = "yyyy"
azure_subscription_id = "zzzz"
azure_tenant_id = "abcd"
/* azure_region = "" */
ssh_public_key = ""

References

I double checked that terraform.tfvars should work as per in issue #6935

[marclennox]

Any luck on solving this? I'm having a similar problem. It looks to me like using provider alias with azurerm simply doesn't work.

[leowmjw]

@marclennox my workaround as above is to use environmental variables. If your case is about provider alias not working, it looks to be a different problem and you should file another issue for it.

[Constantin07]

+1 same issue on CentOS 7.2 with v0.7.6.

[test terraform]$ terraform plan
module root: 1 error(s) occurred:

* provider config 'aws': unknown variable referenced: 'region'. define it with 'variable' blocks

[leowmjw]

@stack72 just a quick update; while trying to understand this problem, I tried to build the azurerm provider separately as a plugin (for the resources I need) --> https://github.com/leowmjw/terraform-provider-fixazurerm and lo, behold terraform.tfvars works!

Could you point me to which parts of the code and especially which test to be run which is related to this? It is clear that when run as a plugin reading variables from terraform.tfvars works fine.

FYI @marclennox @Constantin07 Feel free to build my copy of the fixazurerm (and add your own missing items) plugin if you really need the terraform.tfvars and cannot use the Environment variables to pass in data.

[samirageb]

@leowmjw What I've noticed is that .tfvars files only work with azurerm provider if the variables are also declared (it's ok if they're empty) in the actual .tf file they're used. If you don't have the vars locally declared in the .tf file and only have them declared in the .tfvars, then (for me at least) it wouldn't work.

[leowmjw]

@samirageb Interesting, as that is exactly how I set it up (see below); what version of Terraform are you using and what platform? Are you able to get it working with the latest v0.7.10? If yes I'll be interested to see a gist of a sample that I can try on my end.

Based on my analysis; there was a regression when all the plugins are rolled in to the single binary. As I tested above, when it runs as a plugin it works but I could not find where in the code base where the reading of the terraform.tfvars differs when it is a normal plugin vs a built-in provider :(

main.tf

variable "azure_access_key" {
  description = "Access key for Azure"
}
...

terraform.tfvars

azure_access_key = "a-b-c-d-e-f"
...

[samirageb]

Hm..... @leowmjw I've had it working the past versions since 0.7.7 and can confirm it works for me in 0.7.10 on both Windows & Ubuntu. The only difference I have in my .tf file is that I don't add anything addition to the variable declarations. My .tfvars file would be identical to yours, but my local variables are declared as such (in the tf file):

variable "config_container_name"        { }
variable "config_key"                   { }
variable "config_access_key"            { }

Maybe that'll help?

[samirageb]

@leowmjw I noticed in your provider block you have an alias defined. I don't use that in my provider definition, so that is the only other difference I see with respect to my files.

[leowmjw]

Good catch @samirageb! It was indeed the alias which causes the ResourceConfig to become empty; after removing the "alias" parameter; the "terraform plan" command works with the terraform.tfvars!

Unfortunately, I still could not figure out where it breaks down; but looks to be very early in the process after reading the HCL (which seems to parse and fill in the Alias value correctly).

azurerm-provider (internal) 2016/11/14 14:09:12 ResourceConfig: &{ComputedKeys:[] Raw:map[] Config:map[] raw:<nil>}
azurerm-provider (internal) 2016/11/14 14:09:12 Schema: map[string]*schema.Schema{"subscription_id":*schema.Schema{Type:4, Optional:false, Required:true, DiffSuppressFunc:(schema.SchemaDiffSuppressFunc)(nil), Default:interface {}(nil), DefaultFunc:(schema.SchemaDefaultFunc)(0xc7a410), Description:"", InputDefault:"", Computed:false, ForceNew:false, StateFunc:(schema.SchemaStateFunc)(nil), Elem:interface {}(nil), MaxItems:0, MinItems:0, Set:(schema.SchemaSetFunc)(nil), ComputedWhen:[]string(nil), ConflictsWith:[]string(nil), Deprecated:"", Removed:"", ValidateFunc:(schema.SchemaValidateFunc)(nil), Sensitive:false}, "client_id":*schema.Schema{Type:4, Optional:false, Required:true, DiffSuppressFunc:(schema.SchemaDiffSuppressFunc)(nil), Default:interface {}(nil), DefaultFunc:(schema.SchemaDefaultFunc)(0xc7a410), Description:"", InputDefault:"", Computed:false, ForceNew:false, StateFunc:(schema.SchemaStateFunc)(nil), Elem:interface {}(nil), MaxItems:0, MinItems:0, Set:(schema.SchemaSetFunc)(nil), ComputedWhen:[]string(nil), ConflictsWith:[]string(nil), Deprecated:"", Removed:"", ValidateFunc:(schema.SchemaValidateFunc)(nil), Sensitive:false}, "client_secret":*schema.Schema{Type:4, Optional:false, Required:true, DiffSuppressFunc:(schema.SchemaDiffSuppressFunc)(nil), Default:interface {}(nil), DefaultFunc:(schema.SchemaDefaultFunc)(0xc7a410), Description:"", InputDefault:"", Computed:false, ForceNew:false, StateFunc:(schema.SchemaStateFunc)(nil), Elem:interface {}(nil), MaxItems:0, MinItems:0, Set:(schema.SchemaSetFunc)(nil), ComputedWhen:[]string(nil), ConflictsWith:[]string(nil), Deprecated:"", Removed:"", ValidateFunc:(schema.SchemaValidateFunc)(nil), Sensitive:false}, "tenant_id":*schema.Schema{Type:4, Optional:false, Required:true, DiffSuppressFunc:(schema.SchemaDiffSuppressFunc)(nil), Default:interface {}(nil), DefaultFunc:(schema.SchemaDefaultFunc)(0xc7a410), Description:"", InputDefault:"", Computed:false, ForceNew:false, StateFunc:(schema.SchemaStateFunc)(nil), Elem:interface {}(nil), MaxItems:0, MinItems:0, Set:(schema.SchemaSetFunc)(nil), ComputedWhen:[]string(nil), ConflictsWith:[]string(nil), Deprecated:"", Removed:"", ValidateFunc:(schema.SchemaValidateFunc)(nil), Sensitive:false}} SchemaMap: map[subscription_id:0x425678a50 client_id:0x425678b40 client_secret:0x425678c30 tenant_id:0x425678d20]
azurerm-provider (internal) 2016/11/14 14:09:12 Err: <nil> Diff: <nil>
azurerm-provider (internal) 2016/11/14 14:09:12 Err: <nil> Data: &{schema:map[subscription_id:0x425678a50 client_id:0x425678b40 client_secret:0x425678c30 tenant_id:0x425678d20] config:<nil> state:<nil> diff:<nil> meta:map[] multiReader:<nil> setWriter:<nil> newState:<nil> partial:false partialMap:map[] once:{m:{state:0 sema:0} done:0} isNew:false}
azurerm-provider (internal) 2016/11/14 14:09:12 Err: *multierror.Error{Errors:[]error{(*errors.errorString)(0x425647b40), (*errors.errorString)(0x425647b60), (*errors.errorString)(0x425647b70), (*errors.errorString)(0x425647b80)}, ErrorFormat:(multierror.ErrorFormatFunc)(nil)} Meta: <nil>

[samirageb]

@leowmjw Can you verify that you're executing the command in a directory that doesn't have a .terraform directory in it? I wonder if there's a .tfstate that might be causing this issue... Also, try running just straight 'terraform apply' instead of terraform plan and post back results.

[leowmjw]

@samirageb Here you go: latest version of Terraform binary, no extraneous .terraform directories and all tfstate removed.
I have a ~/.terraformrc (see content further down at the end) but I don't think it is relevant.

leow$ ~/TERRAFORM/terraform --version
Terraform v0.7.10

leow$ ls -al
total 32
drwxr-xr-x  6 leow  staff   204 Nov 14 17:12 .
drwxr-xr-x  4 leow  staff   136 Oct 27 20:27 ..
-rw-r--r--  1 leow  staff  3555 Nov 14 17:11 main.tf
-rw-r--r--  1 leow  staff   660 Oct 31 23:47 outputs.tf
-rw-r--r--  1 leow  staff   466 Oct 27 13:03 terraform.tfvars
-rw-r--r--  1 leow  staff   134 Aug  1 22:53 terraform.tfvars.example

Fails with the "alias" parameter in the azurerm provider:

leow$ grep alias main.tf 
  alias = "bob"
leow$ ~/TERRAFORM/terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but
will not be persisted to local or remote state storage.

Error refreshing state: 1 error(s) occurred:

* 4 error(s) occurred:

* Subscription ID must be configured for the AzureRM provider
* Client ID must be configured for the AzureRM provider
* Client Secret must be configured for the AzureRM provider
* Tenant ID must be configured for the AzureRM provider

Fails with "terraform apply" too :(

leow$ ~/TERRAFORM/terraform apply
Error refreshing state: 1 error(s) occurred:

* 4 error(s) occurred:

* Subscription ID must be configured for the AzureRM provider
* Client ID must be configured for the AzureRM provider
* Client Secret must be configured for the AzureRM provider
* Tenant ID must be configured for the AzureRM provider

When the "alias" parameter is removed:

leow$ grep alias main.tf 
#  alias = "bob"
leow$ ~/TERRAFORM/terraform plan
...
+ azurerm_network_interface.network_interface
    applied_dns_servers.#:                                                  "<computed>"
    dns_servers.#:                                                          "<computed>"
    enable_ip_forwarding:                                                   "false"
    internal_dns_name_label:                                                "<computed>"
    internal_fqdn:                                                          "<computed>"
    ip_configuration.#:                                                     "1"
    ip_configuration.~1191762289.load_balancer_backend_address_pools_ids.#: "<computed>"
...

Content of /.terraformrc

leow$ cat ~/.terraformrc 
providers {
    fixazurerm = "/Users/leow/Desktop/PROJECTS/GOLANG/src/github.com/leowmjw/terraform-provider-fixazurerm/terraform-provider-fixazurerm"

    lxd = "/Users/leow/Desktop/PROJECTS/GOLANG/src/github.com/sl1pm4t/terraform-provider-lxd/terraform-provider-lxd"

}

[samirageb]

@leowmjw I was hoping you'd show what happened if you removed the alias & did an apply. (curious to see if that would work)
At this point it looks like you've somehow changed the .tf file originally posted on this thread. My only suggestion would be to try a single resource and try to get it provisioned. I suspect that the problem you're running into is resource-specific. I can confirm that I'm able to provision network_interfaces, public IPs, and virtual machines with Terraform. I have yet to unfortunately try anything beyond that.

My suggestion would be to just try to get a network interface to be created with the ext tfvars and build from there. I can guarantee that should work.

[whiskeyjay]

@leowmjw My understanding of the issue is that once you specify the "alias" field in provider, the provider is no longer the "default" provider for resources. Thus you'll have to specify "provider" in resources in order to use a specific aliased provider.

In your example tf file in the original post, resources "development" and "network" should all have a field:

  provider = "azurerm.sinar_proj"

Otherwise a default azurerm provider will be used, which is not configured with needed information in your tf file, hence the error message.

Just my two cents.

[leowmjw]

Before I forget to update again; @whiskeyjay you hit it right on target! You need to specify another provider for the "default" and choose the aliased provider when needed. Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.