Terraform-provider-digitalocean: droplet: Inconsistent plan output from 1.0.0 to 1.0.2

[aymanrady]

Terraform Version

```diff
Terraform v0.11.8

• provider.acme v1.0.1
• provider.cloudflare v1.3.0
  
  
  
  • 
    
    
    • provider.digitalocean v1.0.0
    
    
    
  
  
  • 
    
    
    • provider.digitalocean v1.0.2
    
    
    
  
  
• provider.google v1.17.1
• provider.local v1.1.0
• provider.mailgun v0.1.0
• provider.null v1.0.0
• provider.random v2.0.0
• provider.template v1.0.0
• provider.tls v1.2.0
  ```

Affected Resource(s)

• digitalocean_droplet

Terraform Configuration Files

resource "digitalocean_droplet" "server" {
  name       = "${var.server_name}"
  image      = "ubuntu-18-04-x64"
  region     = "${var.server_region}"
  size       = "${var.server_size}"
  backups    = "${var.server_backups_enabled}"
  monitoring = "${var.server_monitoring_enabled}"
  user_data  = "${data.template_file.cloud_config.rendered}"
  tags       = ["terraform-managed"]

  lifecycle {
    ignore_changes = [
      "user_data",
    ]
  }
}

Debug Output

Unfortunately, I can't provide a trace

Expected Behavior

Same plan output

Actual Behavior

• v1.0.0
  ```
  Refreshing Terraform state in-memory prior to plan...
  The refreshed state will be used to calculate this plan, but will not be
  persisted to local or remote state storage.

  digitalocean_droplet.server: Refreshing state... (ID: )

---

No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, no
actions need to be performed.
```

• v1.0.2
  ```
  Refreshing Terraform state in-memory prior to plan...
  The refreshed state will be used to calculate this plan, but will not be
  persisted to local or remote state storage.

  digitalocean_droplet.server: Refreshing state... (ID: )

---

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

-/+ module.one.digitalocean_droplet.server (new resource required)
id: "" => (forces new resource)
image: "37208325" => "ubuntu-18-04-x64" (forces new resource)

---

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
```

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform plan with v1.0.0
2. terraform plan with v1.0.2

Important Factoids

• $ doctl compute droplet get <redacted> -o json | jq '.[].image'

  {
  "id": 37208325,
  "name": "18.04 x64",
  "type": "snapshot",
  "distribution": "Ubuntu",
  "regions": [
    "nyc1",
    "sfo1",
    "nyc2",
    "ams2",
    "sgp1",
    "lon1",
    "nyc3",
    "ams3",
    "fra1",
    "tor1",
    "sfo2",
    "blr1"
  ],
  "min_disk_size": 20,
  "created_at": "2018-08-14T20:03:35Z"
  }
  

• $ doctl compute image get ubuntu-18-04-x64 -o json

  [
  {
    "id": 38897365,
    "name": "18.04 x64",
    "type": "snapshot",
    "distribution": "Ubuntu",
    "slug": "ubuntu-18-04-x64",
    "public": true,
    "regions": [
      "nyc1",
      "sfo1",
      "nyc2",
      "ams2",
      "sgp1",
      "lon1",
      "nyc3",
      "ams3",
      "fra1",
      "tor1",
      "sfo2",
      "blr1"
    ],
    "min_disk_size": 20,
    "created_at": "2018-10-05T16:44:52Z"
  }
  ]
  

• default.tfstate
  { "version": 3, "terraform_version": "0.11.8", "serial": 90, "lineage": "d9a9709a-052f-1397-8026-d913d84d1e8e", "modules": [ { "path": [ "root" ], "outputs": {}, "resources": {"<redacted>"}, "depends_on": [] }, { "path": [ "root", "one" ], "outputs": {}, "resources": { "digitalocean_droplet.server": { "type": "digitalocean_droplet", "depends_on": [ "data.template_file.cloud_config" ], "primary": { "id": "<redacted>", "attributes": { "id": "<redacted>", "image": "ubuntu-18-04-x64", <redacted> }, "meta": { "schema_version": "1" }, "tainted": false }, "deposed": [], "provider": "provider.digitalocean" } }, "depends_on": [] } ] }

Notes

• Ran terraform on two machines ( ubuntu 16.04 & 18.04 )
• I checked the changes between v1.0.0 & v1.0.2 and couldn't find a reason for this behavior
• At the time of server creation, I suspect that digitalocean was returning a slug in the api response but no longer does as the image id is not associated with the latest distribution image available

[caarlos0]

i was using 1.0.2 for a couple of days already, and today while running a terraform plan, it proposed to recreate all machines due to image changes.

It seems to be something on the DO API :/

[caarlos0]

the workaround is basically to change the image to whatever ID the plan says it is changing from.

very annoying.

[eddiezane]

@aymanrady I am unable to reproduce what you're seeing. Can you please provide a minimal, copy pastable, and verified not working config that I can use?

My steps so far have been:

1. git checkout e560dd6814c0118d8c65c99a8576585fc80e4afe (1.0.0 release)
2. Build and install to ~/.terraform.d/plugins
3. terraform init
4. terraform apply
5. git checkout 24d95a1ec42e05a46a10053af1224838254075e6 (1.0.2 release)
6. Repeat 2-4
7. No changes in plan

[caarlos0]

TBH I don't believe the problem is on the provider itself... it happens for me on both versions... with machines launched manually and imported and with machines launched with terraform...

I believe DO's changes some API response, and some instances are returning the image id instead of the image name, making the state inconsistent...

[eddiezane]

@caarlos0 do you have specifics I can dig into?

the workaround is basically to change the image to whatever ID the plan says it is changing from.

Maybe shoot me an email with the ID's from the plan here? My username @digitalocean.com.

[caarlos0]

sure, will do @eddiezane !

[aymanrady]

@eddiezane I can't provide a reproducible test case as it appears by changes in digitalocean api response over time, but I'll try to explain how I think it happens.

using this configuration:

resource "digitalocean_droplet" "server" {
  name   = "server"
  image  = "ubuntu-18-04-x64"
  region = "nyc1"
  size   = "s-1vcpu-1gb"
}

terraform apply will submit a godo.DropletCreateRequest with godo.Image.Slug set
https://github.com/terraform-providers/terraform-provider-digitalocean/blob/20da9cd4e9536597c8cd0f8ad661e001ae4f6a70/digitalocean/resource_digitalocean_droplet.go#L208-L214

and store the slug returned in tfstate
https://github.com/terraform-providers/terraform-provider-digitalocean/blob/20da9cd4e9536597c8cd0f8ad661e001ae4f6a70/digitalocean/resource_digitalocean_droplet.go#L304-L311

this works great as the resource uses a slug and godo.Droplets.Get returns a slug in the response.

sometime passes, Digitalocean updated their official distribution image and moved the "ubuntu-18-04-x64" to point to another image, while the droplet created above stores a reference to an old image ( in digitalocean ) which is no longer identified by the slug.

next terrafrom apply fires godo.Droplets.Get, the response doesn't contain an image slug and we trip this condition
https://github.com/terraform-providers/terraform-provider-digitalocean/blob/20da9cd4e9536597c8cd0f8ad661e001ae4f6a70/digitalocean/resource_digitalocean_droplet.go#L305-L308

terrafrom now thinks we changed the image when we didn't, DigitalOcean just changed their response.

I hope I made it clearer, If this doesn't clear it up I'll look into pointing you at one of our servers which has this problem

[kajogo777]

will this fix be released anytime soon?

[caarlos0]

happened again today to like 100 droplets.

spend half an hour "fixing" my terraform config.

[RuslanZavacky]

Curious if I've had similar issue, but just killed ~10 droplets. Was just adding new records for domain and did not expect Terraform to drop all droplets :D I am so glad that it wasn't a production env.. even though, dropping Gitlab instance was a bit of pain.

I was on version 1.1.0 though.

[caarlos0]

I was on version 1.1.0 though.

it doesn't really matter which version it seems...

[RuslanZavacky]

Yeap, @aymanrady explanation is 100% whats happening.. it does seem, that if this adapter can validate not only against slug, but also and ID, that would help.

image:  "4xxxxxx1" => "ubuntu-18-04-x64" (forces new resource)

This is what I've just got to recently created droplet, provider suggested to kill it again.. :)

@andrewsomething maybe you have some ideas about it? Thanks in advance!

[ManneW]

Hi!

Would a possible workaround be to add some lifecycle configuration to ignore the change of the image field?

Something along these lines:

resource "digitalocean_droplet" "my_droplet" {
  image              = "some-valid-slug"
  ...
  lifecycle {
    ignore_changes = ["image"]
  }
}

Or are there any downsides (except for the case when you actually want to change the image) to this approach?

In the case of a "desired re-creation due to image change", an adaption might be to move the actual ignore_changes value to a variable to make it easier to un-ignore it when you want to make the change.

[RuslanZavacky]

Thats a good suggestion @ManneW. While there will be no better way, I will use this one :) Thank you.

[benjamin-maynard]

@andrewsomething Wondering if you have any update on this? Quite a critical bug and makes it very difficult to use this in production.

[andrewsomething]

A fix for this issue should be available in v1.12.0. In order to upgrade an existing project to the latest version, run:

terraform init -upgrade

You can find the full changelog here: https://github.com/terraform-providers/terraform-provider-digitalocean/blob/master/CHANGELOG.md#120-april-23-2019

[trietphm]

Hi,
I'm using v1.22.2 and having this issue, Terraform forces me to replace the droplet with a new image id. Happen to field user_data as well.

$ terraform providers
.
├── provider.digitalocean ~> 1.22.2

$  terraform plan
# module.xyz must be replaced
-/+ resource "digitalocean_droplet" "xyz" {
     ~ image                = "538xxxxx" -> "ubuntu-18-04-x64" # forces replacement
     + user_data            = "3152exxxx" # forces replacement
     ...
}

Not sure if it's because getting droplet info from API has no slug attribute in image

$ doctl compute droplet get xyz -o json

{
...
   "image": {
      "id": 538xxxxx,
      "name": "18.04.3 (LTS) x64",
      "type": "base",
      "distribution": "Ubuntu",
      "min_disk_size": 20,
      "size_gigabytes": 2.36,
      "created_at": "2019-10-22T01:38:19Z",
      "description": "Ubuntu 18.04 x64 20191022",
      "status": "deleted"
    },
}