Terraform: How to set undefined value

[joshuaspence]

I am writing a module for creating EC2 instances. We have a lot of duplication between our EC2 instance configuration (tags, user data, common security groups, etc) and creating a module eliminates code duplication. I am not sure, however, how to set a value to be undefined. For example, suppose that my module looked like this:

variable "availability_zone" {
  type = "string"
  default = ""
}

resource "aws_instance" "instance" {
  availability_zone = "${var.availability_zone}"
}

My question here is what is the correct default value for the availability_zone variable such that it is essential an optional parameter? Empty string seems to work in some cases, but I'm not sure if this would work in all cases? Basically I am looking for the Terraform equivalent of puppet's undef.

[tmlink]

I could really use this too.
The vSphere provider's "vsphere_virtual_machine" resource "network_interface" block has a ipv4_prefix_length is numberic and therefore a default of "" gives cannot parse '' as int: strconv.ParseInt: parsing "": invalid syntax

So I can't write modules that allow either static or dynamic addresses. It's one or the other.

[elocnatsirt]

Very interested in having the ability to set undefined variables. I am in the process of writing modules now, and it would be nice to be able to set optional values as undefined by default so that the module could contain everything the resource has to offer by default but not pulled in if it wasn't necessary.

[mitchellh]

Most resources at the moment must just be written to accept some "zero" value as unset. If a resource doesn't support this please report a bug. We hope that in time we'll have a way to unset a value but at the moment we do not.

[joshuaspence]

I'm curious... if this functionality does not currently exist and you do hope to implement it one day, then why close this issue? I ask out of curiosity rather than frustration. Is the reason that Hashicorp prefer only to use GitHub issues for tracking features that can be placed somewhere on the short-term roadmap rather than the long-term road map? If so, is there any place to record longer-term feature requests? Or is there no attempt to capture these feature requests because Terraform itself is moving so rapidly?

[dgolja]

Resurrecting this conversation.

Most resources at the moment must just be written to accept some "zero" value as unset.

What are the "zero" - unset values for the parameters ? For example I would like to know the "zero" values for the resource aws_db_instance, particularly the parameter allocated_storage ? This parameter is optional unless a snapshot_identifier or replicate_source_db is provided. The documentation doens't define the zero/undef value.

I would like to have a module that allow either an allocated_storage size or replicate_source_db to be used.

So something along there lines.

resource "aws_db_instance" "db" {
    ...
    allocated_storage = "${var.replicate_source_db == "none" ? var.allocated_storage : 0 }"
    replicate_source_db = "${var.replicate_source_db != "none" ? var.replicate_source_db : 0}"
    ...
}

If some of the parameters have precedence (example: replicate_source_db > snapshot_identifier > allocated_storage) would be great to document it.

[BijeshSamson11]

Hi,
I face an issue with terraform. I have several environments in Azure, some are having load balacers and some are not having load balancers. I made a very generic code for VM deployment, which will automatically add the VM's to load balancer backend pool when VM's are provisioned.
But using same code when I need to deploy VM's are not using load balancer, I need to manually comment out (#load_balancer_backend_address_pools_ids) in main file. Is there any way how to set undefined value (null value) in the "environment.tfvars" file, so that I don't need to change the "Main" file everytime?

Following option I tried, but doesn't help me. Any help greatly appreciated

main.ft file contains following

resource "azurerm_network_interface" "nic" {
name = "${var.nicname}${count.index +1}"
location = "${var.location}"
resource_group_name = "${var.azurerm_resource_group}"
count = "${var.count}"
ip_configuration {
name = "ipconfig${count.index}"
subnet_id = "${var.subnet_id}"
private_ip_address_allocation = "Dynamic"
load_balancer_backend_address_pools_ids = ["${var.backend_pool_ids}"]

variable.tf file contains following

variable "backend_pool_ids" {}

environment.tfvars file contains following

backend_pool_ids = ""

[AndrewFarley]

So, I'm trying to develop some really dynamic modules, and there are SO many variables which do not support this "zero" to unset concept, and some elements in terraform that should actually accept 0 because 0 is a valid value for that parameter. This approach and the reason this bug was closed doesn't make sense, are we, the community supposed to go open 100 different bugs for each variable we run into (I've found 5 casually just now in addition to the 5 bugs linked above) and just get them fixed as we need them? This seems extremely retroactive and pessimistic of a design. I don't understand why we can't have something similar to...

resource "object" "name" {
   name = "my object name"
   optional_value = "${var.somevar == "production" ? 300 : null}"
}

I'm seeing a recurring theme in my almost two years of using terraform, that something like this above is absolutely necessary to create modern modules without having to create tons of different messy optional count-based resources with all the different types of optional values I might or might not set that don't suppor this "0 unset" concept. Would love some further thinking and feedback on this topic please. Especially with the recent terraform module registry going live, something like this is becoming more and more critical to implement (just like getting count support on modules is as well).

Nuances (aka, Flaws) like this really drive my clients away from wanting to adopt Terraform. Because of this flaw, I'm having to maintain multiple slightly modified copies of the same module. Anytime that is necessary usually indicates there is something wrong with the tool/language I'm using.

[gretel]

i agree with @AndrewFarley - feel like hashicrop is trying to 'manage' the not-so-good feedback that counterfeits the initial hype of the tool we talk about.
having hundreds of circular issues is what i personally have observed in regards to other projects just before people started switching to more suitable (modern) approaches.

[mattolenik]

I can't understand how this was closed. At present it is impossible to pass undefined vars into modules. E.g. my module wants to pass some variable foo to some AWS resource, for example. That resource's variable is optional, I can either define it in my HCL or not. In my module, I MUST define that variable, because its value will be passed into the module. But now I can't leave it "unset." I MUST set it to some value, be that 0 or empty string. But when I use 0, I get validation errors from AWS because it's supposed to be UNSET, not a "default" value. Saying this is a bug in the provider is just passing the buck. Who decides what is "default"? You'll get different "defaults" across providers. It forces everyone to implement a 0/empty string check in every resource rather than just having native support for a mechanism in TF.

It's really frustrating that issues like this seem to keep getting closed. It's like Hashicorp just doesn't want to hear about it and would rather put their fingers in their ears and assume it isn't their problem.

Imagine this were a programming language. It's like saying you don't want null or option types and everyone should pass around magic strings and numbers to indicate no value, and everyone should check for those values on their own, rather than having the language provide a feature. It is forcing everyone to do a check over and over again when it should really just be done once within TF. TF should not _force_ you to pass variables around when they are undefined. That's just downright silly. If you make providers fix this, it becomes a neverending battle of trying to get every provider to implement a fix for this quirk.

[AndrewFarley]

Yeah, welcome to the club @mattolenik With modules being at the forefront of the future of Terraform, absolutely this is one of those pivotal "duh" moments where we need the language to evolve to support this. Anytime I do anything even remotely advanced with Terraform I run into this bug and it frustrates the hell out of me. Right now for one client I'm justifying having to have 3 versions of the same module in increasingly "creative" ways because of this bug. Can we raise a bounty to fix this or something? What will it take? :P

[mattolenik]

I understand it not being implemented just yet, and that's okay. What bothers me is why these keep getting closed.

[mattolenik]

This is also affected by this: https://github.com/terraform-providers/terraform-provider-aws/issues/2063

Assuming "-1" for a null value is just insane. This isn't even something providers CAN fix because -1 is a valid value, so there's no way to tell if it's "null" or not.

[clumsysnake]

I've had exactly the same issues.

IMO we need to be able to write real code to define the graph. Either using a new terraform-specific configuration language, or some kind of HCL templating that is closed over the variables and data sources, or give up on HCL-json compatibility and extend HCL, or some other idea. Is there a feature roadmap or RFC where this is atleast being thought about? I understand terraform is still young and things take time to mature, but I'm worried that the terraform team doesn't think the limtiations of HCL are a problem.

When I first started with terraform I remember the hope being that HCL itself would be "kinda basically just json" and 3rd parties would develop HCL-generation tools in front. Well those never really happened in a meaningful way, and IMO since we've been adding piecemeal a mishmash of functions that are incomplete and often unintuitive to use, or pushing the burden onto provider repos as-needed and with uncoordinated semantics. Now with backends and workspaces and data sources and modules, code-generation can't work, if it ever could have.

Addressing this is important for adoption. I'm beach-heading terraform within our client, a large multinational, and I'm at the point now where I have to pitch their internal ops department to adopt what i've done (and hopefully buy TF Enterprise!) All the issues with inflexibility in conditionalizing resource arguments impairs that pitch. Perhaps experts (like the gruntwork.io team) can get around the issues through extended trickery. But neither sharp learning curves nor "just write some wrapper scripts" endears.

[rahulkp220]

@BijeshSamson11 seems like you can asnwer https://stackoverflow.com/questions/48306089/how-do-i-make-a-field-optional-inside-a-terraform-resource if possible?

[rafaelmagu]

Another use case where undefined would've worked well:

Terraform will perform the following actions:

  ~ module.mongo-test.google_compute_instance.mongo[0]
      metadata.%:              "1" => "2"
      metadata.startup-script: "" => ""

  ~ module.mongo-test.google_compute_instance.mongo[1]
      metadata.%:              "1" => "2"
      metadata.startup-script: "" => ""

  ~ module.mongo-test.google_compute_instance.mongo[2]
      metadata.%:              "1" => "2"
      metadata.startup-script: "" => ""

This is silly.

[cornfeedhobo]

@mitchellh Any hope you could print out your comment and make it an office banner or something?
I run into this issue constantly. If I filed a bug request for resources that don't gracefully handle empty values, it would be for half of the AWS resources, _at the least_.

For example, I imagine that most of the logic dealing with tags is repetitive, yet every resource I have worked with will fail on an empty map. That seems fairly fundamental.

[lvicentesanchez]

I'm having this problem too... I have a kafka module and sometimes I want to pass an instance profile, but others I don't care and so far it's all or nothing.

[cornfeedhobo]

@lvicentesanchez yeah, you have to make two versions of the resource, with a conditional on whether an instance profile is passed. it's messy and repetitive.

[rahulkp220]

@cornfeedhobo Yeah, I also had to do the same thing in my case. Seems dumb.

[abferm]

I have a module to create an AWS LB target group, listener rule, and dns record and found this issue because I was trying to make listener_rule_priority optional. I really don't want to make two different "resources" and use some messy "count" options to switch which one is created.

[AndrewFarley]

This is the state of things unfortunately. There was too much internal resistance to modifying HCL to support conditionals and now we all suffer because of it. I manage 5 of the same module with various options enabled or disabled that can not be made into a single module because of the lack of this bug and/or the lack of conditionals. There is a much rumored update coming to terraform to add conditionals which will solve this. We will all have to wait impatiently. :)

[mllanes]

Same here with a CloudFront distribution were I want to optionally set a web_acl. One thing I found over and over again with terraform is that for some reason, doesn't fit the mold of the "software engineer" mindset; case in point, principles like DRY are difficult to achieve in the current state of affairs.

[AnthonyWC]

It would be great if Terraform has a way to represent null value; and at least for AWS provider it should able to differentiate between empty and null input.

In the case of Cloudformation, AWS added the pseudo parameter AWS::NoValue. In conjunction with Cloudformation Condition Functions Fn::If, it allows one to use NoValue as a return value to remove the property itself.

e.g. in cloudformation:

Fn::If: [condition_name, value_if_true, value_if_false]

CloudFormation removes the DBSnapshotIdentifier property if condition evaluates to false.

 DBSnapshotIdentifier:
      !If [UseDBSnapshot, !Ref DBSnapshotName, !Ref "AWS::NoValue"]

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-conditions.html#intrinsic-function-reference-conditions-if

[AnthonyWC]

At the resource level, one can use the count to achieve some level of conditional check but it's limited at the resource level, not individual parameter level., e.g. https://github.com/hashicorp/terraform/issues/15281

[cornfeedhobo]

@AnthonyWC you should read this comment

[AnthonyWC]

I've read it but that was made more than a year ago.

[anderssoder]

Having the same problem and needing to maintain several versions of a module to support different use cases just becuase I can't unset optional settings or remove whole configuration blocks within a resource when they should not be present.

Really frustrating! Terraform needs to add better support for this if it's to become a mature product, and I do hope so because besides this I like it. But it's just not feasible having to maintain several module versions instead of one dynamic one.

@mitchellh When is the team going to prioritize and focus on this? The community is really behind this happening as can be seen by the comments here.

[amardomingo]

Same problem here. I need to conditionally set one of two mutually exclusive variables, and could really use a null/undefined value

[cornfeedhobo]

@AnthonyWC lol, yeah I didn't say it was a solution. And @mitchellh is notorious for muting threads once he's made a judgement. This is a dead-in-the-water request it appears. My guess is they are trying to solve it globally, probably when the make hcl-terraform v2.

[AnthonyWC]

Ok so one workaround at the resource level via count would be to do something like this:

variable "optional_resource_parameter" {
  default = "default_null"
}

resource "foo" "bar" {
  count = "${var.optional_resource_parameter == "default_null" ? 0 : 1}"
}

The trick is to define a default value to represent null and then use count to set to 0 if the default is not changed to anything else. This still require you to write out multiple version of resource level definition though so it's still not ideal but at least your module can reference the same directory.

[anderssoder]

@AnthonyWC Yeah that's what I'm doing to toggle whole resources in the module and it works well, think it's been mentioned before in the comments here also. Have for example like this

resource "aws_cloudfront_distribution" "default" {
  viewer_certificate {
acm_certificate_arn            = "${var.acm_certificate_arn == "" ? join(" ",aws_acm_certificate_validation.cert.*.certificate_arn) : var.acm_certificate_arn}"
  }
}

resource "aws_acm_certificate" "cert" {
  count = "${var.acm_certificate_arn == "" ? 1 : 0}"
}

For a module which creates cloudfront dist, r53 records and accompanying acm certs. If you already have a cert created you provide a cert arn in the acm_certificate_arn variable which is optional, so if not provided it will create the required resources. Just showing on of them in the snippet above.

The issue for me is not toggling resources, it's about toggling fields and configuration blocks within a resource. The other day we got a use case where a team wanted to use Lambda@Edge. For cloudfront resource that's a specific config block which associates the lambda function with the distribution. I can't have the module always creating this association, only when it's needed. So had to branch out and have that as a separate version. That's one use case, there are many...

@apparentlymart @mitchellh Some attention to this, please?

[phinze]

To the 20 folks who are watching this issue - native null value support is coming in Terraform 0.12! The HCL2 integration effort is well underway.

This description from @apparentlymart is from last year but is still mostly accurate in its description of the incoming functionality: https://github.com/hashicorp/terraform/issues/14037#issuecomment-341726807

[midacts]

Terraform v12.0 does not have any end date in sight.
It has already been 6+ months since the v0.12 preview was announced.

This issue has been open for 2.5+ years. : [

[rcarmo]

Well, I hope it's gonna be a great 2019 (for me, at least, having this will help make it a better year).

[dene14]

It's not gonna happen in my life...

[sprutner]

0.12 is out!

[stefan-rapp]

https://github.com/hashicorp/terraform/releases/tag/v0.12.0 --> Solves the problem finally! :-)

[wontonst]

The value is just null

eg var.foo != "" ? var.foo : null

[mikalai-t]

But it seems doesn't work as expected :( Just called from the upstream module and passed null value as described at Terraform 0.12 docs expecting the value would be omitted

Error: Error in function call

  on ../aws-iam-role/main.tf line 20, in locals:
  20:   iam_role_tags                       = "${merge(var.iam_role_tags, map("Name", format("%s", local.iam_role_name)), map("terraform", "true"))}"
    |----------------
    | local.iam_role_name is "rds-config-role"
    | var.iam_role_tags is null

Call to function "merge" failed: panic in function implementation: can't use
ElementIterator on null value
goroutine 325 [running]:
runtime/debug.Stack(0xc00066dfa8, 0x1856a20, 0x1f7f5d0)

[pselle]

@mikalai-t Thank you for the report here! I've opened a fresh issue as this looks like a valid bug we should fix. I'm going to lock the conversation here, since the original issue has been closed by the release of 0.12, and I want to make sure we don't lose track of comments on this open issue.