Kops: Feature Flag Request: Do not try to delete AWS Subnets

[michaelajr]

We bring our own VPC. I need to tag the subnets with KubernetesCluster so ELBs will add them to their zones list. When I delete a cluster, Kops always tries to delete the subnets. It would be great if there was a feature flag to disable this.

[chrislovecnm]

@michaelajr actually I will up you one :) I do not think kops should delete them :P Can you provide details on how to recreate, and what functionality would you like??

[jkinkead]

We want this as well. The KubernetesCluster tag is used by the AWS driver to determine candidate subnets for ELBs. I'm guessing that this is used indirectly by kops to determine the subnets in the cluster.

What I'd like to see is two features:

1. kops only deletes subnets that are marked for deletion. Marking is done in a user-transparent way (i.e. by resource tag or field in the spec).
2. kops marks which subnets it creates as candidates for deletion.

The first is much more important than the second, although the second would be pretty great.

There was a suggestion in issue #1220 that kops should handle permission tags on resources, which is interesting.

[michaelajr]

example-1.6-us-east-1.txt

@chrislovecnm To recreate, create a cluster via Kops using and an exiting VPC. Attached is some example YAML that will need to be tweaked with your own SubnetsIDs, Zone Identitfier, DNS names, etc. etc. etc.

Then, tag the subnets in accordance to the what the AWS ELB integration needs KubernetesCluster: CLUSTER_NAME

Then try to delete the cluster.

You will see that kpos tries to delete the subnets. It would be great if Kops knew it did not create the VPC and therefor ignore all VPC resources it did not specifically create.

[jkinkead]

FWIW, Kubernetes 1.6.0 has a new tag format.

[michaelajr]

@jkinkead I could not find any documentation on the new tagging structure. Wondering if adding kubernetes.io/cluster/CLUSTER_NAME without a value or without the value being owned will 1) work with kops where it will not attempt to delete the subnet, and 2) work with the AWS-ELB integration where it will add the subnet to the ELB's zones. Guess I can set up a test.

[jkinkead]

An empty value works with ELB integration on Kubernetes 1.6.0. From what I can tell, the value is currently unchecked, but owned looks to be reserved for future use.

kops (1.6.0-alpha2) ignores subnets with this tag, but I wouldn't count on that behavior in the future.

[chrislovecnm]

@michaelajr I am experiencing problems on a dev branch that the VPC is being deleted completely. We need to put in logic to check the cluster spec, and if the ids are listed in the cluster spec, leave the asset alone.

[chrislovecnm]

To be clear we are picking up naming of the tags. So if you use the exact kops tags on the VPC it will delete the VPC. We need to implement a check for checking the VPC id and the subnet ids. If those exist in the configuration we should not delete those resources.

[robertojrojas]

I'm working on this issue.

[chrislovecnm]

/assign

I have another PR in which is based on @robertojrojas' PR, and double checks that we do not delete many user shared items.

[chrislovecnm]

With the correct tags, this should be fixed in master. Closing