Cloudformation-coverage-roadmap: AWS::ECS::Cluster-CapacityProvider

[alexhall]

1. Title

AWS::ECS::Cluster-CapacityProvider

2. Scope of request

AWS recently announced the availability of Capacity Providers for ECS clusters to manage scaling on EC2 autoscaling groups. We would like to use this new feature in our CloudFormation templates.

3. Expected behavior

When creating or updating an AWS::ECS::Cluster resource, it should be possible to specify a CapacityProvider property that, if set, will be used to manage scale-up/scale-down of an EC2 autoscaling group. Removing CapacityProvider from an existing Cluster should disable managed scaling of the EC2 autoscaling group.

I don't have strong opinions on whether the CapacityProvider property should be specified as a reference to a new AWS::ECS::CapacityProvider resource, or inline as part of the Cluster resource.

4. Suggest specific test cases

5. Helpful Links to speed up research and evaluation

Blog post: https://aws.amazon.com/about-aws/whats-new/2019/12/amazon-ecs-cluster-auto-scaling-now-available/
User guide: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cluster-auto-scaling.html
API docs: https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CapacityProvider.html

6. Category

Compute

7. Any additional context (optional)

[rclark]

I do feel strongly that this should be implemented as a new resource type for AWS::ECS::CapacityProvider. We should be able to reference those resources in a CapacityProviders property of an AWS::ECS::Cluster.

[softprops]

I'm very interested in using the new fargate spot capacity provider for reducing our aws bill. We manage all of our our ECS infra through cloudformation so not having this feature is a blocker for us

[dhoeric]

aws-cli already support on specifying FARGATE_SPOT & FARGATE on --capacity-providers.
That might be helpful on integrating with cloudformation :)

ref: https://docs.aws.amazon.com/cli/latest/reference/ecs/create-cluster.html

[josephbmanley]

I feel like this is currently blocked internally for the CFN team because you are unable to delete capacity providers. See https://github.com/aws/containers-roadmap/issues/632

@dhoeric If you look at the CLI commands for capacity providers, you'll find:

• create-capacity-provider
• describe-capacity-providers
• put-cluster-capacity-providers

but no way to detach or delete the capacity provider which is a roadblock even for creating a custom resource.

[wmcewan]

upvote

[JohnPreston]

aws-cli already support on specifying FARGATE_SPOT & FARGATE on --capacity-providers.
That might be helpful on integrating with cloudformation :)

ref: https://docs.aws.amazon.com/cli/latest/reference/ecs/create-cluster.html

Just opened an issue for this
https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/459

[sean-kang]

Do you have ETA for this?

[tschutte]

I feel like this is currently blocked internally for the CFN team because you are unable to delete capacity providers. See aws/containers-roadmap#632

@dhoeric If you look at the CLI commands for capacity providers, you'll find:

* create-capacity-provider

* describe-capacity-providers

* put-cluster-capacity-providers

but no way to detach or delete the capacity provider which is a roadblock even for creating a custom resource.

The same is true of task definitions (cannot delete) and that hasn't stopped CFN from supporting it. So hopefully the team is just slow to implement.

[jsherlockppb]

Issue #239 is also quite relevant for this. Currently, we can't configure instance protection on an Auto Scaling Group in Cloudformation, which is a requirement for enabling Managed Termination Protection in CAS. Point 10 in the Capacity Provider developer guide states:

When managed termination protection is enabled, Amazon ECS prevents Amazon EC2 instances that contain tasks and that are in an Auto Scaling group from being terminated during a scale-in action. Managed termination protection can only be enabled if the Auto Scaling group also has instance protection from scale in enabled and if managed scaling is enabled.

It is only possible to enable managed termination protection if the underlying auto scaling group is _already_ configured to enable termination protection, so even if we have the ability to provision a Capacity Provider via Cloudformation, we won't be able to enable Managed Termination Protection if the underlying auto scaling group is also provisioned via Cloudformation (ego, without termination protection), until issue 239 is resolved.

[luiseduardocolon]

Right now I am expecting to see this in the next 6 weeks. Stay tuned. :)

[JohnPreston]

@luiseduardocolon Great news on #301.
How is that going to impact #459
It would be nice in the current circumstances to be able to define the default strategy already given FARGATE and FARGATE_SPOT are here by default.

[mwarkentin]

Looks like this is just about here (not actually seeing it in the docs yet though): https://twitter.com/aws_doc/status/1273943424849383424

[mwarkentin]

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-capacityprovider.html

[rhlarora84]

I do not see any update to Autoscaling Resource yet. Unless that is updated to support instance protection scale in, we can't use the termination protection on capacity provider. I hope that is coming soon.

[JohnPreston]

How does one get the services in the cluster to use the cluster capacity provider strategy by default?
Seems like there is a specific API call for services (boto ref)
I had opened https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/459 so, are we hoping to see this done too given the successful release of https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/301 ?
Many thanks! You are AWSome!

[manokaran3529]

How does one get the services in the cluster to use the cluster capacity provider strategy by default?
Seems like there is a specific API call for services (boto ref)
I had opened #459 so, are we hoping to see this done too given the successful release of #301 ?
Many thanks! You are AWSome!

Please remove the 'LaunchType' property from your ecs service resource block of your cfn template or add a condition to pass 'AWS::NoValue' then the service would pick the default capacity provider strategy

[JohnPreston]

@manokaran3529 Thanks for the quick response. Happy to find where in the docs make a PR to let people know about it (to update this)