Terraform-provider-google: Weird state error with TF 0.14.0 on changes

[Sytten]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave _+1_ or _me too_ comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Problem

I think there is an issue with terraform 0.14.0 causing bad state to happen on changes.
See https://github.com/hashicorp/terraform-provider-google/issues/7944 for the first time it happened.
I got a similar error with cloud map:

When expanding the plan for google_compute_url_map.default to include new
values learned so far during apply, provider
"registry.terraform.io/hashicorp/google" produced an invalid new value for
.map_id: was cty.NumberIntVal(3.092957057560021842e+18), but now
cty.NumberIntVal(3.092957057560022e+18).

[edwardmedia]

@Sytten Is this the same issue of #7944? Can you share the full debug log?

[Sytten]

Same issue but different resources so it hints at a larger problem. Is it possible to get debug logs on terraform cloud? On the other issue I was able to resolve it by manually removing deposed instances from the state file.

[Sytten]

@edwardmedia Found it, here is the debug of the apply. Let me know if you need the debug of the planning, it contains sensitive stuff so I can send it over email but not here.
run-f3xqCuxgPnmv3xrm-apply-log.txt

[edwardmedia]

@Sytten I noticed you use random data for the name. Any particular reason? Do you want an new name for it every time it runs?

resource "google_compute_managed_ssl_certificate" "default" {
  ...
  name = random_id.cert_name.hex
...
}

[Sytten]

@edwardmedia I use the keeper parameter with the list of domains, its because you need a new name for the provider to create a new certificate when you change domains in the cert. So this doesnt change on every run.

[edwardmedia]

@Sytten can you post the other resources? It looks this happens in the following applies. If this is the case, can you specify the steps how I can repro the issue? You may redact the secrets in the logs and post them at https://gist.github.com/.

[Sytten]

@edwardmedia I am still unconvinced that has anything to do with the random provider. The logs I posted above are related only to this resource which doesn't use it.

resource "google_compute_url_map" "default" {
  project = var.gcp_project

  name        = "${module.this.id}-https"
  description = "URL map for load balancing"

  # Default
  default_service = local.services.service_marketing
  default_route_action {
    url_rewrite {
      host_rewrite = "REDACTED"
    }
  }

  # Marketing
  host_rule {
    hosts        = ["REDACTED.io"]
    path_matcher = "marketing"
  }

  path_matcher {
    name            = "marketing"
    default_service = local.services.service_marketing

    default_route_action {
      url_rewrite {
        host_rewrite = "REDACTED.gr8.com"
      }
    }
  }

  # CDN
  host_rule {
    hosts        = ["cdn.REDACTED.io"]
    path_matcher = "cdn"
  }

  path_matcher {
    name            = "cdn"
    default_service = local.storage.bucket_cdn
  }
}

I think the steps are fairly simple:

1. Use terraform 0.14.0 with latest gcp provider version
2. Create the url map like above
3. make a change to the url map
4. Try to apply it, it fails

It seems to happen on any in-place change of basically any resource.
Here are the plan logs: https://gist.github.com/Sytten/3fa690a85698642a44035dadf1458b4b

[edwardmedia]

@Sytten can you specify what change(s) in step3? To repro, I don't know the data for variables/ locals / modules, etc. Can you hard-coded all the values, and rerun all the steps? Basically I need all code that I can repro the issue

make a change to the url map

[Sytten]

Anything can change it doesn't matter, the issue is not in the resource AFAIK. I can create a repo to reproduce if you want. Do the tests even pass with TF 0.14? This is such a core issue I would assume they don't.

[edwardmedia]

@Sytten there are daily tests which do not show failures. I need specific steps and changes to repro the issue.

[renchap]

I am seeing the same issue with TF 0.14.0 & provider 3.50.0, and I am using fixed strings as names:

When expanding the plan for google_compute_url_map.public_assets to include
new values learned so far during apply, provider
"registry.terraform.io/hashicorp/google" produced an invalid new value for
.map_id: was cty.NumberIntVal(4.39984439957665894e+18), but now
cty.NumberIntVal(4.399844399576659e+18).

[Sytten]

@renchap Did you see similar issues with other resources? because I had issues with another resource in #7944 so thats why I thought it was a more serious issue.

[jpuyy]

we have the same issue

Error: Provider produced inconsistent final plan
When expanding the plan for google_compute_url_map.default[0] to include new
values learned so far during apply, provider
"registry.terraform.io/hashicorp/google" produced an invalid new value for
.map_id: was cty.NumberIntVal(1.25906097628084157e+18), but now
cty.NumberIntVal(1.2590609762808413e+18).

[slevenick]

This looks like a problem in 0.14.0 rather than the provider. I have filed https://github.com/hashicorp/terraform/issues/27208

I would recommend downgrading to 0.13.x if possible as we track down what the fix should be

[Abdelwaheb-Hnaien]

I'm having the same issue with google_compute_url_map resource since I upgraded to terraform v0.14.0
I created the url_map in my project using terraform v0.13.5
what I'm trying to do now is to make the url_map point to a new backend service (previously an instance group, now a bucket) and then I get this error:

Error: Provider produced inconsistent final plan


When expanding the plan for
module.google_compute_url_map_uat-mcm-url-map.google_compute_url_map.urlmap to
include new values learned so far during apply, provider
"registry.terraform.io/hashicorp/google" produced an invalid new value for
.map_id: was cty.NumberIntVal(6.438816093867301036e+18), but now
cty.NumberIntVal(6.438816093867301e+18).


This is a bug in the provider, which should be reported in the provider's own
issue tracker. 

I also tried to do the same with terraform v0.14.3, but it didn't work also.

This action (changing the backend) was working perfectly with terraform version 0.13.5

[thilohaas]

This should be resolved with TF2.4.0 https://github.com/hashicorp/terraform-plugin-sdk/issues/655#issuecomment-748389946 but seems like the Google provider needs to be updated to reflect the fix.

[slevenick]

We have a fix for this using the new version of the SDK. This should be in the release expected to go out next week.

[slevenick]

This should be available in 3.51.1

[Sytten]

@slevenick Can you update the changelog, this release got past under the radar. I didn't receive a github notification for it.