Terraform-provider-google: google_service_networking_connection error: Cannot modify allocated ranges in CreateConnection. Please use UpdateConnection

[stingus]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment
• If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

Terraform v0.11.13

• provider.google v2.2.0
• provider.google-beta v2.2.0

Affected Resource(s)

• google_service_networking_connection

Terraform Configuration Files

resource "google_compute_network" "network" {
  name                    = "network"
  project                 = "${var.project_id}"
  auto_create_subnetworks = false
}

resource "google_compute_global_address" "private_ip_range" {
  provider      = "google-beta"
  name          = "private-ip-range-stage"
  purpose       = "VPC_PEERING"
  address_type  = "INTERNAL"
  prefix_length = 16
  network       = "${google_compute_network.network.self_link}"
}

resource "google_service_networking_connection" "service_connection" {
  provider                = "google-beta"
  network                 = "${google_compute_network.network.self_link}"
  service                 = "servicenetworking.googleapis.com"
  reserved_peering_ranges = ["${google_compute_global_address.private_ip_range.name}"]
}

Debug Output

https://gist.github.com/stingus/5534ca7d45d22c3e23b2f0fe9aa0dd68

Panic Output

Expected Behavior

A google_service_networking_connection resource is created

Actual Behavior

Error received: google_service_networking_connection.service_connection: Error waiting for Create Service Networking Connection: Error code 9, message: Cannot modify allocated ranges in CreateConnection. Please use UpdateConnection.

Steps to Reproduce

1. terraform apply

Important Factoids

It seems that the google_compute_global_address.name should be a dash separated version of the resource name. In the above config, the resource name is "private_ip_range":

resource "google_compute_global_address" "private_ip_range" {
...

The google_service_networking_connection resource is created IF the google_compute_global_address name is:

resource "google_compute_global_address" "private_ip_range" {
  name          = "private-ip-range"
  ...

If the name contains private-ip-ranges (notice the trailing s) then it fails. Same behavior applies for google_compute_network when used with google_service_networking_connection.

References

[emilymye]

I was able to create the resources without issue, so I'm not really sure what is happening. Can you make sure that you don't have other connections using the same reserved_peering_ranges?

google_compute_global_address.name should be a dash separated version of the resource name

Which resource name are you refering to?

[stingus]

@emilymye yes, I'm positive it's the only connection using that range.

[stonith]

I suspect there's a bug in GCP where creation of these resources works the first time but does not work after deleting and trying to recreate. I get the same error message even in the web GCP console when trying to recreate these resources after successfully creating them previously.

[joeslazaro]

I ran into the same Terraform error message when I was trying to create a Cloud SQL instance with private IP access only. I also reproduced the original error in the console like @stonith and found a related issue in Google's issue tracker in which post 11 says "we've identified the issue that caused Cloud SQL instances to get IP addresses from a previous allocated range".

Using a tip from one of the posts, I was able to manually fix the VPC peering and get it to show up correctly in the console with this gcloud command:

gcloud beta services vpc-peerings update \
    --service=servicenetworking.googleapis.com \
    --ranges=private-ip-address \
    --network=sandbox-net \
    --project=sandbox-XXXXX \
    --force

Next I did a terraform apply and it said that it was able to create the google_service_networking_connection resource successfully.

Last I did a terraform destroy followed by a terraform apply and it was able to re-create all of the networking resources with no errors.

It seems now like something had gotten stuck and needed that update command to clear it.

[sermilrod]

Facing the same issue. Unfortunately manual fix for me is not an option.

Are there any new developments to fix this?

[ciiqr]

@sermilrod google support sent me this issue tracker link and mentioned that there's currently no estimate on how long it will take to fix. https://issuetracker.google.com/issues/131908322

[t4nec]

I faced same issue, and after removing s from name of vpc connection, It successfully deployed all resources.

[vdboor]

Workaround I've found.

First create it with an empty range like this:

resource "google_service_networking_connection" "service_connection" {
  provider                = "google-beta"
  network                 = "${google_compute_network.network.self_link}"
  service                 = "servicenetworking.googleapis.com"
  reserved_peering_ranges = []
}

then update it with the updated configuration:

resource "google_service_networking_connection" "service_connection" {
  provider                = "google-beta"
  network                 = "${google_compute_network.network.self_link}"
  service                 = "servicenetworking.googleapis.com"
  reserved_peering_ranges = ["${google_compute_global_address.private_ip_range.name}"]
}

[dizer]

Another workaround for 0.12:

Try to create google_service_networking_connection with reserved_peering_ranges = []. And run tf plan. It will show autogenerated google_compute_global_address. In my case it was equal to "google-managed-services-${var.vpc_network_name}". So describing google_compute_global_address resource with such name solved issue for me.

[k0ffee]

If everyone uses "update --force" as a workaround, why not put that into the terraform provider itself, to be able to use it in automated environments again? I've tested this diff for creating a single Postgres instance and for creating two Postgres instances in two clusters inside the same project (the latter resulted before in: "Error 9, use UpdateConnection"):

diff --git a/google-beta/resource_service_networking_connection.go b/google-beta/resource_service_networking_connection.go
index 224c93e2..bf01d41b 100644
--- a/google-beta/resource_service_networking_connection.go
+++ b/google-beta/resource_service_networking_connection.go
@@ -69,7 +69,21 @@ func resourceServiceNetworkingConnectionCreate(d *schema.ResourceData, meta inte
    }

    parentService := formatParentService(d.Get("service").(string))
-   op, err := config.clientServiceNetworking.Services.Connections.Create(parentService, connection).Do()
+   // We use Patch instead of Create, because we're getting
+   //  "Error waiting for Create Service Networking Connection:
+   //   Error code 9, message: Cannot modify allocated ranges in
+   //   CreateConnection. Please use UpdateConnection."
+   // if we're creating peerings to more than one VPC (like two
+   // CloudSQL instances within one project, peered with two
+   // clusters.)
+   //
+   // This is a workaround for:
+   // https://issuetracker.google.com/issues/131908322
+   //
+   // The API docs don't specify that you can do connections/-,
+   // but that's what gcloud does, and it's easier than grabbing
+   // the connection name.
+   op, err := config.clientServiceNetworking.Services.Connections.Patch(parentService+"/connections/-", connection).UpdateMask("reservedPeeringRanges").Force(true).Do()
    if err != nil {
        return err
    }

See: https://github.com/terraform-providers/terraform-provider-google-beta/pull/1240

[davidepedranz]

Any news on this? The PR has been there for one month already now 🤔

[slevenick]

The PR fixing this was merged in and should be released soon. Upgrading to 3.1.0 when it is released should fix this issue

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!