Terraform: Tag names containing dots are always deleted

[gar1t]

This tags definition:

tags {                                                                      
    "foo.bar" = "123"                                                       
}

Will cause an existing tag named "foo.bar" to be deleted. terraform apply always shows this:

tags.foo.bar: "<existing value>" => ""

[mitchellh]

There is another issue tracking this, but the unfortunate thing is that periods in tag names simply can't be supported with the current state format. It'd be a silly dance to do so. We'll add an error.

We're working on fixing this but in the short term could you not use dots? :( sorry.

[gar1t]

-Shakes fist at universe- ;)

Yeah, unfortunately we're using tag names from another application and making that change would be very hard indeed as it's managing servers in independent contexts.

I'm wondering about an escaping mechanism in your names that could be used to specify unicode characters, which could be passed along to AWS [1] - crazy?

[1] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/Using_Tags.html

[gar1t]

One other thought here - if Terraform didn't insist on its tags being the entire tagset, this would not be an issue either. Terraform could concede that other applications may use tags and act only to ensure that its tagset is a correct subset. If someone wants to delete a tag explicitly, they can specify the name with an empty string.

To maintain compatibility with the current regime, you might call this thing "ExtraTags" (or similar).

[tony612]

:+1:

[pdf]

This limitation of the parser has wider implications too - if you need to pass keys containing periods to a provider, you have to make users do some dance where they replace periods with some other character - that must not be reserved, or overlap with any other valid characters - and then replace them before sending to the backend.

I'd really like to have quoted keys in maps treated as literals (syntactically preferable IMO), or failing that, allow periods to be escaped via \., but I don't know how realistic either option is.

[marekrogala]

:+1:
pretty important to me

[tony612]

I just wonder is there any progress for this problem? Dot is a convention for the names of our tags, so it's not easy to change :(

[ghost]

:thumbsup:
Also breaks out tag name convention based on dots...

[bigkraig]

:+1: this is important for the api gateway

[coen-hyde]

As linked above this is breaking advanced_options for AWS's Elasticsearch service. Would be great to get this fixed.

[nicolai86]

@mitchellh any guidance on how we can support hashicorp in solving this issues?

[jen20]

I _think_ #6322 should fix this, though I've not tested anyspecific cases. I'll add some tests around this issue and #6471 to the dev-0.7 branch and see what we can do. At minimum, the actual limitation is possible to remove now the internal representation of maps has changed, so this should be possible to support now.

[nicolai86]

@jen20 that sounds great. this is a missing piece in a great aws_api_gateway implementation, so any progress on this front makes my day, literally, b/c I can start improving terraform again :)

[jen20]

@nicolai86 I just verified that at least in the simple case it works. I'll test with actual resources shortly and close up this issue by adding some tests if all is well.

[sodre]

@jen20 I am also trying to fix this issue for PR #6115. Can you point how you would expect the provider code would have to change in view for #6322 ?

[BillBrady]

Any news on this issue? A customer has several . in their tag names as a company wide convention and it is a worry for them. Any updates appreciated greatly.
Thanks ~Bill

[justinsb]

I just tested the 0.7-rc release, and this is not fixed. Is there a new syntax I should be using?

Here is a simple test case:

resource "aws_ebs_volume" "myvolume" {
  availability_zone = "us-east-1b"
  size = 20
  type = "gp2"

  tags = {
    "a.b" = "c"
  }
}

terraform --version
Terraform v0.7.0-rc1 (301da85f30239e87b30db254a25706a6d41c2522)

If the parser can't be fixed, a workaround is to do what other AWS resources (e.g. autoscaling groups) do. They can be tagged arbitrarily, because instead of a map they use a list of objects [{key="a.b", value="c"} ... ].

[jen20]

As of #6322 this has been fixed - 0.7.0-rc1 does not exhibit this behaviour:

Configuration:

provider "aws" {
    region = "us-west-2"
}

variable "tags" {
    default = {
        Name = "Test"
        Environment = "Production"
        Test.With.Dot = "Test Value"
    }
}

resource "aws_vpc" "test" {
    cidr_block = "10.0.0.0/16"

    tags = "${var.tags}"
}

Output:

$ terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but
will not be persisted to local or remote state storage.


The Terraform execution plan has been generated and is shown below.
Resources are shown in alphabetical order for quick scanning. Green resources
will be created (or destroyed and then created if an existing resource
exists), yellow resources are being changed in-place, and red resources
will be destroyed. Cyan entries are data sources to be read.

Note: You didn't specify an "-out" parameter to save this plan, so when
"apply" is called, Terraform can't guarantee this is what will execute.

+ aws_vpc.test
    cidr_block:                "10.0.0.0/16"
    default_network_acl_id:    "<computed>"
    default_security_group_id: "<computed>"
    dhcp_options_id:           "<computed>"
    enable_classiclink:        "<computed>"
    enable_dns_hostnames:      "<computed>"
    enable_dns_support:        "<computed>"
    instance_tenancy:          "<computed>"
    main_route_table_id:       "<computed>"
    tags.%:                    "3"
    tags.Environment:          "Production"
    tags.Name:                 "Test"
    tags.Test.With.Dot:        "Test Value"


Plan: 1 to add, 0 to change, 0 to destroy.

$ terraform apply
aws_vpc.test: Creating...
  cidr_block:                "" => "10.0.0.0/16"
  default_network_acl_id:    "" => "<computed>"
  default_security_group_id: "" => "<computed>"
  dhcp_options_id:           "" => "<computed>"
  enable_classiclink:        "" => "<computed>"
  enable_dns_hostnames:      "" => "<computed>"
  enable_dns_support:        "" => "<computed>"
  instance_tenancy:          "" => "<computed>"
  main_route_table_id:       "" => "<computed>"
  tags.%:                    "" => "3"
  tags.Environment:          "" => "Production"
  tags.Name:                 "" => "Test"
  tags.Test.With.Dot:        "" => "Test Value"
aws_vpc.test: Creation complete

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.

State path: terraform.tfstate


$ aws ec2 describe-vpcs --region us-west-2
{
    "Vpcs": [
        {
            "VpcId": "vpc-cafb69ae",
            "InstanceTenancy": "default",
            "Tags": [
                {
                    "Value": "Test Value",
                    "Key": "Test.With.Dot"
                },
                {
                    "Value": "Test",
                    "Key": "Name"
                },
                {
                    "Value": "Production",
                    "Key": "Environment"
                }
            ],
            // Unnecessary output removed

[sodre]

@jen20, can you verify if the code still works if you write

provider "aws" {
    region = "us-west-2"
}
resource "aws_vpc" "test" {
    cidr_block = "10.0.0.0/16"

    tags = {
        Name = "Test"
        Environment = "Production"
        Test.With.Dot = "Test Value"
    }
}

I was not able to get it to work with the triton provider. I just want to make sure it is not an issue on how that provider code is parsing the tags.

[mitchellh]

@sodre I believe you have to wrap fields with dots in double quotes, but I'm not certain.

[sodre]

@mitchellh With quotes it still does not work.

[jen20]

Hi @sodre - the code you posted works for me (with a build from master). I have not tried with the Triton provider - it's possible there is something in there preventing it working. If you have a repro could you open a new issue about this? Thanks!

[sodre]

@jen20 Thank you, I am working on a AccTest for using "dots" in the Triton provider. I will continue posting it to PR #6115.

[tony612]

@jen20 Hi, sodre's code doesn't work for me (Terraform 0.7.0-rc2):

$ terraform plan

+ aws_vpc.test
    cidr_block:                "" => "10.0.0.0/16"
    default_network_acl_id:    "" => "<computed>"
    default_security_group_id: "" => "<computed>"
    dhcp_options_id:           "" => "<computed>"
    enable_classiclink:        "" => "<computed>"
    enable_dns_hostnames:      "" => "<computed>"
    enable_dns_support:        "" => "<computed>"
    instance_tenancy:          "" => "<computed>"
    main_route_table_id:       "" => "<computed>"
    tags.#:                    "" => "3"
    tags.Environment:          "" => "Production"
    tags.Name:                 "" => "Test"
    tags.Test.With.Dot:        "" => ""


Plan: 1 to add, 0 to change, 0 to destroy.

And it still doesn't work with quotes

[justinsb]

Can we please reopen? This is still not fixed, even building from source:

resource "aws_ebs_volume" "myvolume" {
  availability_zone = "us-east-1b"
  size = 20
  type = "gp2"

  tags = {
    "a.b" = "c"
  }
}

plan gives:

+ aws_ebs_volume.myvolume
    availability_zone: "us-east-1b"
    encrypted:         "<computed>"
    iops:              "<computed>"
    kms_key_id:        "<computed>"
    size:              "20"
    snapshot_id:       "<computed>"
    tags.%:            "1"
    tags.a.b:          ""
    type:              "gp2"

[sodre]

@stack72,
Can you investigate this issue again? It seems I am not the only one who has not been able to replicate the fix that @jen20 mentioned.

Also, note that there is an open PR #7130 that shows the issue is not fixed at least the triton provider. We could enrich the PR to include the VPC test by @tony612 and EBS test by @justinsb.

[tony612]

It works now 👍

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.