Terraform-provider-google: when creating a private cluster, master_authorized_network is always enabled

[joshkurz]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment
• If an issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to "hashibot", a community member has claimed the issue already.

Terraform Version

Terraform v0.11.8

Affected Resource(s)

google_container_cluster

Terraform Configuration Files

resource "google_container_cluster" "default" {
  name               = "${var.cluster-name}"
  region             = "${var.gcp-region}"
  initial_node_count = "${var.inital_node_count}"
  network            = "${var.gcp-network}"
  subnetwork         = "${google_compute_subnetwork.gke-subnet.self_link}"
  min_master_version = "${var.master_version != "" ? var.master_version : data.google_container_engine_versions.default.latest_master_version}"
  provider = "google-beta"
  project = "${var.gcp-project}"

  ip_allocation_policy {
    cluster_secondary_range_name = "pod-tier"
    services_secondary_range_name = "service-tier"
  }

  maintenance_policy {
    daily_maintenance_window {
        start_time = "03:00"
    }
  }

  addons_config {
    http_load_balancing {
      disabled = true
    }
    horizontal_pod_autoscaling {
      disabled = true
    }
    kubernetes_dashboard {
      disabled = true
    }
  }

  private_cluster_config {
    enable_private_endpoint = false
    enable_private_nodes = true
    master_ipv4_cidr_block = "192.168.128.0/28"
  }

  network_policy {
    enabled = true
  }
  node_config {
    oauth_scopes = [
      "https://www.googleapis.com/auth/compute",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
    ]

    labels {
      group = "${var.cluster-name}"
    }

    tags = ["${var.cluster-name}"]
    machine_type = "${var.machine_type}"
  }

  // Wait for the GCE LB controller to cleanup the resources.
  provisioner "local-exec" {
    when    = "destroy"
    command = "sleep 90"
  }
}

Debug Output

The cluster gets built fine.

Expected Behavior

Master authorized networks should be disabled.

Actual Behavior

Master authorized networks is enabled.

Steps to Reproduce

run the above terraform template.

1. terraform apply

Important Factoids

I tried a few different setups to get this to work. I tried adding an empty config, I tried setting enabled = false, which bombed out as it's not in the spec.

[calinah]

I'm getting the same so I tried manually updating the field in the UI (not ideal) and then reapplying to attempt to get the tfstate in a good place. This is what I get after I update the field manually on the UI and running a plan:

google_container_cluster.new_container_cluster: Refreshing state... (ID: cluster-2)

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  ~ module.new-gke-cluster.google_container_cluster.new_container_cluster
      master_authorized_networks_config.#: "1" => "0"


Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.new-gke-cluster.google_container_cluster.new_container_cluster: Modifying... (ID: cluster-2)
  master_authorized_networks_config.#: "1" => "0"

Error: Error applying plan:

1 error(s) occurred:

* module.new-gke-cluster.google_container_cluster.new_container_cluster: 1 error(s) occurred:

* google_container_cluster.new_container_cluster: googleapi: Error 400: Must specify a field to update., badRequest

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

I was using terraform 0.11.7 but upgraded to 0.11.10 to rule out version related issues. Can we get an update on this?

@joshkurz Have you managed to find a workaround for this?

[calinah]

For anyone who came across this issue one workaround is to set master_authorized_networks_config to 0.0.0.0/0 the public internet CIDR, by adding this block in your private cluster config:

master_authorized_networks_config = {
    cidr_blocks = [
      {
        cidr_block   = "0.0.0.0/0"
        display_name = "any-name"
      },
    ]
  }

[wyardley]

I think master authorized networks are required for private clusters when enable_private_endpoint is false. So maybe this is an issue that could be improved with better validation.

[alanwds]

@wyardley i just test enable_private_endpoint setted as true and the master_authorized_networks_config still enabled.

[nimmaj]

I wonder if this issue is related to: https://issuetracker.google.com/issues/123071694

[weeco]

This is also an issue for existing clusters which have been imported in terraform. Every time you apply a change to that cluster, this will reenable the authorized networks.

Terraform expects a master_authorized_networks_config {} block and if you remove that block it will show a change (as expected). However it can never successfully apply this change:

Error: googleapi: Error 400: Must specify a field to update., badRequest

provider "google-beta" {
  version = "~> 2.14"
}
terraform 0.12.6

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!