Terraform: RequestLimitExceeded while refreshing 500/1000 aws_instance instances

[darron]

Lots of output snipped - but here's the relevant lines:

aws_instance.client.4: Refreshing state... (ID: i-c2243318)
aws_instance.client.111: Refreshing state... (ID: i-f124332b)
aws_instance.client.838: Refreshing state... (ID: i-622f79ba)
aws_instance.client.598: Refreshing state... (ID: i-172177cf)
aws_instance.client.241: Refreshing state... (ID: i-b53d2a6f)
aws_instance.client.967: Refreshing state... (ID: i-412c7a99)
aws_instance.client.522: Refreshing state... (ID: i-810d5b59)
Error refreshing state: 7 error(s) occurred:

* aws_instance.client.455: RequestLimitExceeded: Request limit exceeded.
    status code: 503, request id:
* aws_instance.client.421: RequestLimitExceeded: Request limit exceeded.
    status code: 503, request id:
* aws_instance.client.473: RequestLimitExceeded: Request limit exceeded.
    status code: 503, request id:
* aws_instance.client.803: RequestLimitExceeded: Request limit exceeded.
    status code: 503, request id:
* aws_instance.client.177: RequestLimitExceeded: Request limit exceeded.
    status code: 503, request id:
* aws_instance.client.894: RequestLimitExceeded: Request limit exceeded.
    status code: 503, request id:
* aws_instance.client.66: RequestLimitExceeded: Request limit exceeded.
    status code: 503, request id:
make: *** [destroy] Error 1

I've repeatedly run into this problem trying to remove some 500 and 1000 node clusters I've been using for a demo. I've seen these issues - but they're closed:

https://github.com/hashicorp/terraform/issues/3579

https://github.com/hashicorp/terraform/issues/1051

Is there anything else I can do other than manually kill them all? Which I'm doing right now so that I'm not paying for these nodes anymore?

I'm using v0.6.14.

Thanks!

Having the ability to spin them up is awesome! It's flawless at that - ripping them down would be nice to be easier/quicker.

[darron]

If you want an easy way to reproduce this - here's the exact code I used to build and deploy the clusters:

https://github.com/darron/kvexpress-demo

I started seeing the problem at 500 nodes.

[stevegula]

I have found I hit this limit even when I'm not doing particularly large infrastructure in AWS (including when I use service that don't have particularly low rate limits like Route 53)

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.