Terraform-provider-aws: aws_instance.volume_tags gets confused when > 1 volume attached

[hashibot]

_This issue was originally opened by @FransUrbo as hashicorp/terraform#14107. It was migrated here as part of the provider split. The original body of the issue is below._

---

It seems the new volume_tags (thanx btw!) gets confused when there's more than one volume attached to an instance.

Terraform Version

0.9.4

Affected Resource(s)

Please list the resources as a list, for example:

• aws_instance

Terraform Configuration Files

resource "aws_instance" "instance" {
  root_block_device {
    volume_type               = "standard"
    volume_size               = 8
  }

  volume_tags {
    Name                      = "my-instance-root"
  }
}

resource "aws_ebs_volume" "instance" {
  tags {
    Name                      = "my-instance-db"
  }
}

resource "aws_volume_attachment" "instance" {
  instance_id                 = "${aws_instance.instance.id}"
  volume_id                   = "${aws_ebs_volume.instance.id}"
  device_name                 = "/dev/sdb"
}

Expected Behavior

Nothing.

Actual Behavior

Every other time it modified the aws_ebs_volume tags and every other on the aws_instance.

Steps to Reproduce

1. terraform apply

[kpumuk]

👍 have the same issue

[rickhlx]

+1

[acutchin-bitpusher]

still seeing this in 0.10.8

[e-carlin]

+1

[MrMojoRisin49]

+1

[ddsdevon]

Still an issue as of terraform 0.11.1
I added a second volume to an existing instance with its own tags, and now aws_instance and aws_ebs_volume take turns performing an update in place to toggle the tags back and forth.

[geekbass]

Upgraded to version 0.11.1 and I am seeing this issue as well.

[ghost]

+1

[SYC1205]

+1

version 0.11.1 still has this issue

[chan-alex]

+1
Seeing this in version 0.11.1

[smastrorocco]

+1
Seeing this in version 0.11.2

[duhaas2015]

I can confirm I'm seeing the same behavior in 0.11.3, adding the ignore for volumes tags to the lifecycle fixes the problem for now

[gavD]

I am still getting this problem

$ terraform --version
Terraform v0.11.7
+ provider.aws v1.7.0

My code (simplified) looks like:

resource "aws_instance" "allinone" {
  ami                     = "${var.ami}"
  subnet_id               = "${var.subnet_id}"

  root_block_device {
    volume_type = "gp2"
  }

  volume_tags {
    MakeSnapshot   = "False"
  }
}

resource "aws_ebs_volume" "myebs" {

  type              = "gp2"
  size              = "${var.db_data_volume_size}"
  encrypted         = true

  tags {
    MakeSnapshot   = "True"
  }

}

I find that the tag MakeSnapshot on myebs vacillates between False and True with every terraform apply. I've tried making the EBS depends_on the EC2 instance and vica versa and it doesn't help.

Hope that is a clear explanation - it would be great to get a fix on this! :)

[roberulloa]

$ terraform --version
Terraform v0.11.7
+ provider.aws v1.18.0

This issue comes up when you use the volume_tags inside aws_instance with the aws_ebs_volume tag option.
volume_tags { Name = "ebs-root-instance" }

If you use only the tag mode on an aws_ebs_volume, works like a charm.

[arunsandu1]

any solution with using _volume_tags_ inside aws_instance, from not overriding ebs volume tags? No issues with EBS volume tagging. I want to give a tag to the instance root volume using _volume_tags_

terraform version:
Terraform v0.11.7

[ddsdevon]

@roberulloa My current use case is that I am using "volume_tags" to ensure the root volume of the instance is properly tagged, and then I am using aws_ebs_volume on a separately created volume to ensure that specific volume is properly tagged.

I have not yet found a way to modify the root volume created by the aws_instance resource, and the instance's own ebs_block_device option does not appear to allow extra/alternate tags.

[mda590]

+1!

Seeing this exact behavior in version:
Terraform v0.11.7

• provider.aws v1.23.0

[n3ph]

Confirming on:

• Terraform v0.11.7
• provider.aws v1.24.0

provider "aws" {
  max_retries = 3
  region      = "eu-central-1"
  profile     = "devops"
}

data "aws_ami" "amzn2_base" {
  most_recent = true

  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }

  owners = ["137112412989"]
}

data "aws_vpc" "default" {
  default = true
}

data "aws_subnet_ids" "default" {
  vpc_id = "${data.aws_vpc.default.id}"
}

data "aws_subnet" "test" {
  id = "${data.aws_subnet_ids.default.ids[0]}"
}

resource "aws_instance" "test" {
  ami           = "${data.aws_ami.amzn2_base.id}"
  subnet_id     = "${data.aws_subnet.test.id}"
  instance_type = "t2.nano"

  root_block_device {
    volume_type = "gp2"
  }

  volume_tags {
    Name = "test1"
  }
}

resource "aws_ebs_volume" "test" {
  type              = "gp2"
  size              = "1"
  encrypted         = true
  availability_zone = "${data.aws_subnet.test.availability_zone}"

  tags {
    Name = "test2"
  }
}

resource "aws_volume_attachment" "test" {
  instance_id = "${aws_instance.test.id}"
  volume_id   = "${aws_ebs_volume.test.id}"
  device_name = "/dev/sdb"
}

21:22 n3ph@mag-xps ~/tmp/community_gardening_terraform ✔  terraform apply
data.aws_vpc.default: Refreshing state...
data.aws_ami.amzn2_base: Refreshing state...
data.aws_subnet_ids.default: Refreshing state...
data.aws_subnet.test: Refreshing state...
aws_instance.test: Refreshing state... (ID: i-064ce7c94874db85c)
aws_ebs_volume.test: Refreshing state... (ID: vol-037621e6930139f90)
aws_volume_attachment.test: Refreshing state... (ID: vai-2991852907)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ aws_ebs_volume.test
      tags.Name: "test1" => "test2"


Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_ebs_volume.test: Modifying... (ID: vol-037621e6930139f90)
  tags.Name: "test1" => "test2"
aws_ebs_volume.test: Modifications complete after 1s (ID: vol-037621e6930139f90)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
21:23 n3ph@mag-xps ~/tmp/community_gardening_terraform ✔  terraform apply
data.aws_vpc.default: Refreshing state...
data.aws_ami.amzn2_base: Refreshing state...
data.aws_subnet_ids.default: Refreshing state...
data.aws_subnet.test: Refreshing state...
aws_ebs_volume.test: Refreshing state... (ID: vol-037621e6930139f90)
aws_instance.test: Refreshing state... (ID: i-064ce7c94874db85c)
aws_volume_attachment.test: Refreshing state... (ID: vai-2991852907)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ aws_instance.test
      volume_tags.Name: "test2" => "test1"


Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_instance.test: Modifying... (ID: i-064ce7c94874db85c)
  volume_tags.Name: "test2" => "test1"
aws_instance.test: Modifications complete after 3s (ID: i-064ce7c94874db85c)

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
21:23 n3ph@mag-xps ~/tmp/community_gardening_terraform ✔  

[tdmalone]

Related tickets: #770 & #884 (possibly all duplicates of the same issue?)

[justin-sunayu]

+1 Getting this while trying to tag my root volume and data volumes

[gnalawade]

same issue observed
[gnalawade@jhost db]$ terraform -v
Terraform v0.11.7

• provider.aws v1.26.0

aws_volume_attachment.sql101-data01-attach: Refreshing state... (ID: vai-1256441354)

---

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place

Terraform will perform the following actions:

~ aws_instance.sql101
volume_tags.Name: "SQL101.PROD-sdf" => "SQL101.PRDO-sda"

Plan: 0 to add, 1 to change, 0 to destroy.

[spommerening]

Same issue here with

Terraform v0.11.7
+ provider.aws v1.29.0

Anyway I found a temporary work-around:

• Create instance and use volume_tags for setting tag of root volume
• Include volume_tags in ignore_changes of instance
• Create and attach additional volumes which can be tagged now

I am very well aware that this work-around will not work for everyone
and that we still need a fix but maybe this helps someone.

[soar]

Same problem with

Terraform v0.11.7
+ provider.aws v1.30.0

[tdmalone]

Hi @soar @gnalawade @justin-sunayu @n3ph @mda590

The issue will still exist until it’s fixed. There’s no need to add additional comments confirming it’s still present - if you’re affected, add a thumbs up reaction to the initial post in this thread instead.

Comments that don’t include additional information such as troubleshooting, workarounds etc add noise for the subscribers and risk being locked by the maintainers. It also makes it more difficult to find discussion of the actual issue.

Thanks!

[vfoucault]

+1

[morokin]

In the meanwhile, year 2019,
terraform --version
Terraform v0.11.13

• provider.aws v2.2.0

Issue still there

[tdmalone]

@morokin Add a plus one on the initial post in this thread instead - Hashicorp report on those to determine which features to work on. As I mentioned in my above comment that you -1'ed, your post just adds needless noise for others in the same position who are subscribed to this thread, and doesn't actually help the issue get fixed or prioritised.

[bkmeneguello]

My suggestion is to add "tags" attribute to both "root_ebs_volume" and "ebs_block_device" while keeping the "volume_tags". This way "volume_tags" could be a fallback when no tags are defined in the previous elements (and to retain compatibility). If someone don't want all instance block devices tagged with "volume_tags" just suppress it and tag blocks individually.

[philiphope]

2 years. How long do we have to wait for this bug to be fixed?

[rifelpet]

Just for reference to anyone wishing to fix this, the problem stems from the fact that when updating volume_tags in place, terraform sets tags on all volume IDs attached to the instance rather than only the volumes created from the original ec2.RunInstances used to create the instance. Terraform is already able to distinguish the root_block_device from other block devices but not distinguishing ebs_block_devices defined in an aws_instance versus a separate aws_ebs_volume resource. The ec2.RunInstances response does contain the volume IDs of the other block devices which could be used to differentiate an ebs_block_device from an aws_ebs_volume, but any fix to track this information and use it would only fix it for instances created with a version of the aws provider containing this fix, not existing instances nor imported instances.

You can confirm this ambiguity with a terraform show. The definition of this aws_instance has a root_block_device block but no ebs_block_device block, rather it has separate aws_ebs_volume and aws_volume_attachment resources. The separate volume resource's volume ID is now stored as an attribute of the aws_instance:

aws_instance.foo:
  id = i-0e2ee1ae34f051b98
  ebs_block_device.# = 1
  ebs_block_device.4138763540.delete_on_termination = false
  ebs_block_device.4138763540.device_name = /dev/xvdq
  ebs_block_device.4138763540.volume_id = vol-07b923a8c02cc95f0
  ebs_block_device.4138763540.volume_size = 750
  ebs_block_device.4138763540.volume_type = gp2
  ebs_optimized = true
  ...
  root_block_device.# = 1
  root_block_device.0.delete_on_termination = true
  root_block_device.0.iops = 0
  root_block_device.0.volume_id = vol-0aed163a5aa8bf9d8
  root_block_device.0.volume_size = 20
  root_block_device.0.volume_type = standard
aws_ebs_volume.foo:
  id = vol-07b923a8c02cc95f0
  ...
aws_volume_attachment.foo:
  id = vai-1985794965
  device_name = /dev/xvdq
  instance_id = i-0e2ee1ae34f051b98
  skip_destroy = true
  volume_id = vol-07b923a8c02cc95f0

It would be pretty straightforward to use the readBlockDevicesFromInstance function to get the root_block_device's volume ID for setVolumeTags but that wouldnt solve the ambiguity with ebs_block_device.

Personally I'm a fan of this partial fix.

Thoughts?

[slessardjr]

I just overwrote the name of all my kubernetes persistent EBS volumes due to this bug. Is there any plan to fix this? I think the suggestion @rifelpet is a perfect middle ground. Unless the provider makes root_volume_tags and ebs_volume_tags separated.

[muhmud]

2 years. How long do we have to wait for this bug to be fixed?

+ another year.

I just started using Terraform, and it's fantastic, but can we not get some traction on this?

The suggestion by @bkmeneguello above seems like it would be easy to put in without breaking anything. In fact, it's what I tried to do intuitively at first before finding it didn't work.

[bkmeneguello]

I think the proper way is to submit a PR, it should not be that hard.

[binlab]

$ terraform version
Terraform v0.12.24
+ provider.aws v2.53.0

the problem still remains

[alanbantuit]

$ terraform --version
Terraform v0.12.26

• provider.aws v2.66.0

still wants to change tags on the non-root volume:

  ~ volume_tags                  = {
      ~ "Backup" = "dev_de_data" -> "dev_de_root"
      ~ "Name"   = "DEV|DE|Data" -> "DEV|DE|Root"
    }

That apparently just 1 line changed pull request has still not been merged.

[thomasbiddle]

Running into this now, still.

Terraform v0.13.3
+ provider registry.terraform.io/-/aws v3.8.0
+ provider registry.terraform.io/hashicorp/aws v3.8.0

Having an option to just ignore all EBS volumes would be the best solution IMO.