Copilot-cli: Any plan to support Scheduled tasks (cron)?

[pcolazurdo]

I'd like to be able to define cron-like and/or CloudWatch Event triggered tasks directly from Copilot. I couldn't find any reference to it in the documentation - is this something you are looking to add somehow? Happy to get involved into any FR if useful

[efekarakus]

Hi @pcolazurdo ! Yeah, we are planning on adding this feature :)

Can you tell us a little bit more on your usecases with scheduled tasks? Do these tasks talk to a database for example, or do you need any specific parameters from the TaskDefinition surfaced?

Thanks!

[pcolazurdo]

Thanks for your reply. My use case is pretty simple. I have a container that will process information from multiple sources and export this to a static web site hosted in S3 - given this config i only need to be able to setup the task and a Task Role that will give the task enough permissions to retrieve and upload data.
This task will be triggered by a cron-like setting and also via a Cloudwatch event triggered when a new version of some S3 files are uploaded
I can think many other use cases though, happy to share those if you think that would be useful.

[kochalex]

For our background jobs use case we would want each job to be a child of an existing service and to run to be a new container all with the same context & configuration as an existing service, but just run the job command on a schedule instead of the what defined to in the Dockerfile's CMD line and listening on a port.

Also, for the scheduling, taking in a cloudwatch events rule as both cron and rate format would be great.

[jonchase]

I'd use this feature as well - I'd like to use CloudWatch time based events to kick off tasks in Fargate. General event support would be great as well, but the scheduled stuff is what I'm really after.

[rwhitten577]

Adding another +1 here. Currently using Fargate scheduled tasks for ETL jobs when Lambda's limitations prevent us from using it. Do need to define roles with access to S3, pull params from SSM, decrypt them with KMS, etc.

[bvtujo]

Great news, everyone! As of Copilot v0.5.0 we now support the ability to create, deploy, and operate resilient Scheduled Jobs with timeout and retries using simple UNIX cron or duration syntax. Jobs launched this way will have access to all the other services in your Copilot environment via service discovery and support the same Addons functionality that services have, making it super easy to do things like query your database service and upload the results into a dedicated S3 bucket or SNS topic.

Scheduled jobs launched with init, deploy, ls, package, and delete commands, with status, show, and logs coming very soon. Take it for a spin and let us know what you think!