Aws-cli: Add container overrides to put-targets for ECS tasks

[gregeinfrank]

Hi,

According to http://docs.aws.amazon.com/cli/latest/reference/events/put-targets.html, you can now pass in EcsParameters in put-targets and pass in a TaskDefinitionArn and TaskCount, but there is no option for container overrides, which seems to be supported in the AWS Console (see screenshot below):

Is there any plan to support this?
Thank you!

[joguSD]

The CLI supports whatever the public API for the service is, so if it's not supported now we would need to wait for the CloudWatch Events service team to add this functionality to their API. You can try contacting AWS Support to see if the service team has any insight on this feature.

[highbass]

Has there been any updates made on this or any plans to release functionality to support this?

[sdesalas]

Hey guys,

This is working by the way, but its pretty badly documented (hint hint @awstools ):

Its just that you have to use the Input parameter (in addition to EcsParameters) and pass in a "containerOverrides" object, ie:

ecs events put-target \
   --rule my-daily-batch \
   --targets "[{ \
      \"Id\": \"$TARGET_ID\", \
      \"Arn\": \"$CLUSTER_ARN\", \
      \"RoleArn\": \"$ROLE_ARN\", \
      \"EcsParameters\": { \"TaskDefinitionArn\": \"$TASK_DEFINITION_ARN\", \"TaskCount\": 1 }, \
      \"Input\": \"{ \\\"containerOverrides\\\": [{ \\\"name\\\": \\\"$CONTAINER_NAME\\\", \\\"command\\\": [\\\"echo\\\", \\\"hello\\\", \\\"override\\\"] }] }\" \
  }]"

The way I worked this out was by adding the rule manually via the web console (which makes it easy), then doing a CLI command ecs events list-targets-by-rule on the rule I just created.

[alexanderdavidpan]

@sdesalas, the command in the array has to be comma separated strings, right? Not just a single string? Can you provide an example of how to do so?

[renperez]

@sdesalas i'm getting invalid json when running this.

aws events put-targets \
--rule my-daily-batch \
--targets "[{ \
\"Id\": \"$TARGET_ID\", \
\"Arn\": \"$CLUSTER_ARN\", \
\"RoleArn\": \"$ROLE_ARN\", \
\"EcsParameters\": { \"TaskDefinitionArn\": \"$TASK_DEFINITION_ARN\", \"TaskCount\": 1 }, \
\"Input\": \"{ \\"containerOverrides\\": [{ \\"name\\": \\"$CONTAINER_NAME\\", \\"command\\": [\\"echo\\", \\"hello\\", \\"override\\"] }] }\" \
}]"

[sdesalas]

@renperez dont forget to check your escape characters inside "Input":, (ie "containerOverrides" etc.

The quotation marks should be using TRIPLE backslashes (\\\) as per my comment above.

It takes a bit of trial and error until you get it working.

[nwsparks]

It can be simplified a bit further than the example above by wrapping the json in a single quote and some escaping inside the input block.

aws events put-targets \
--rule "dev-referrals-timeliness-4am-gmt" \
--targets '[
  {
      "Id": "dev-referrals-timeliness",
      "Arn": "arn:aws:ecs:us-east-1:667646328794:cluster/qio-dev-ecs",
      "RoleArn": "arn:aws:iam::667646328794:role/ECS_Run_Task_Role",
      "Input": "{ \"containerOverrides\": [{ \"name\": \"dev-data-services\", \"command\": [\"echo\", \"hello\", \"override2\"] }] }",
      "EcsParameters": {
          "TaskDefinitionArn": "arn:aws:ecs:us-east-1:667646328794:task-definition/dev-data-services:115",
          "TaskCount": 1
      }
  }
]'

[sterichards]

I wish CloudFormation would add container overrides to AWS::Events::Rule.....

This becomes a whole lot simpler and less of a hack job when/if they do

[whereisaaron]

Woah, this is badly documented for CloudFormation AWS::Events::Rule too. Nowhere does it mention you can use Inputs in this way. I read the documentation multiple times as I was confused that the console could do overrides but it seemed like CloudFormation could not (nor the CLI). Now I realize it all hinged around Inputs. In the AWS::Events::Rule there is not a single example of using Inputs - for any purpose.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-events-rule.html

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-events-rule-ecsparameters.html

Neither does the EventBridge documentation bring it up or have even a single example. It covers inputs only in the context of something else generating it. And in the context of transforming an input something else generated.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-events-rule.html

@sdesalas saves the day 👍

@sterichards The CloudFormation AWS::Events::Rule Target has an Inputs property I think you can use to pass the overrides is the same manner as the CLI example.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-events-rule-target.html#cfn-events-rule-target-input

[alexisdray]

My issue for this is that you can not both specify inputs and leave the revision empty.

We are using an empty revision specification, which ensures that the tasks TaskDefinitionArn is updated to the latest revision. However it then ignores input overrides and uses whatever is specified in the original task definition. Does anyone know a workaround to this?

[spezam]

Since I found this discussion useful to solve the integration EventBridge -> Fargate, I can tell that it's also possible to use the InputTransformer: in order to pass for example information about the event to the container via containerOverrides.

An example, using Cloudformation (which can be easily translated into cli):

  SomeEventListener:
    Type: AWS::Events::Rule
    Properties:
      ...
      Targets:
        - Id: some-target
           ...
          InputTransformer:
            InputPathsMap:
              source: $.source
            InputTemplate: |
            {
                "containerOverrides": [
                  {
                    "name": "some-container",
                    "environment": [
                      {
                        "name": "EVENT_SOURCE",
                        "value": <source>
                      },
                    ]
                  }
                ]
              }

According to the documentation about transforming target input:

https://docs.aws.amazon.com/eventbridge/latest/userguide/transform-input.html

it's possible to use some reserved variables:
aws.events.rule-arn — The Amazon Resource Name (ARN) of the EventBridge rule.
aws.events.rule-name — The Name of the EventBridge rule.
aws.events.event — A copy of the original event.

I was interested in particular about aws.events.event, to be able to pass the whole event as env variable. But it seems there's something missing there, as the container is not starting (possibly because the override JSON object is malformed.

Hope this extra information will come useful to someone.