Kops: Double check NAT gateway delete code

[justinsb]

This was reported in slack:

kops delete cluster foobar --yes
inserting nat-... to be deleted
inserting nat-... to be deleted
inserting nat-... to be deleted

error from DescribeNatGateways: NatGatewayNotFound: NAT gateway nat-.... was not found
        status code: 400, request id: 6....

[gauravarora]

I'm stuck at this step too. I deleted the NAT gateway by mistake and now I can't delete the cluster because kops errors out. Perhaps a --force flag would help.

[chrislovecnm]

@gauravarora if you do an update, it should recreate the NAT gateways, then do a delete. As always please test before running in production. What version of kops are you using?

[thereallukl]

I just hit this issue with kops 1.7.1. I wasn't able to neither update cluster nor delete. After manual removal of entries related to nat gateway in VPCs route tables I was able to succesfully delete cluster.

[krogon-dp]

NAT Gateways are not tagged. That's why NATs are detected via existing private route or via subnet.

Not sure it was historically impossible, but now it is available (checked at AWS): http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-nat-gateway.html#nat-gateway-tagging

This could solve some problems seen here.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[pulpbill]

Hi, I'm having this issue:

W0107 14:22:17.467080 1362 executor.go:130] error running task "ElasticIP/us-east-1a.my.domain" (9m55s remaining to succeed): error finding AssociatedNatGatewayRouteTable: error listing NatGateway %!q(*string=0xc420c7f108): NatGatewayNotFound: NAT gateway nat-083300682d9a0fa74 was not found status code: 400, request id: 168747d0-8681-4370-b8bb-58a4568ab432
Kops version:
Version 1.10.0 (git-8b52ea6d1)

Can't delete nor update the cluster.