Datadog-agent: Add internal Fargate container to ignore list

[ayozemr]

Describe what happened:
When using Datadog Agent as sidecar container on Fargate (https://www.datadoghq.com/blog/monitor-aws-fargate/), a hidden internal container is discovered by Datadog called "internalecspause" with image "fg-proxy:tinyproxy"

Describe what you expected:
This container should be filtered like its being done in Kubernetes (AFAIK), because its counting for billing and its a hidden container in AWS. Don't think nobody wants to metric it in DD.

Steps to reproduce the issue:
Add Datadog Agent to the task definition of your service with ECS_FARGATE=true and other env vars. When task starts, you will see 3 containers in DD: Your service + DD Agent + AWS internal pause

Additional environment details (Operating System, Cloud provider, etc):
AWS Fargate

Datadog agent task env vars:
DD_AC_EXCLUDE | "image:."
DD_AC_INCLUDE | "image:REDACTED.dkr.ecr.us-east-1.amazonaws.com/."
DD_API_KEY | APIKEY
DD_APM_ENABLED | true
DD_APM_NON_LOCAL_TRAFFIC | true
DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL | true
DD_LOGS_ENABLED | true
ECS_FARGATE | true

[danjamesmay]

You've posted your API key :1st_place_medal:

[ayozemr]

It was just a test one, but thanks anyway...

[jpbochi]

It would be even better if we could have a whitelist (or blacklist) of containers to look for (or ignore).

[ayozemr]

I found an ignore env var, but didnt work for me.

key: DD_AC_EXCLUDE
value: image:fg-proxy image:datadog/agent

[irabinovitch]

@ayozemr Please note that containers in Fargate do not count towards billing at this time. We do plan to release billing for Fargate soon, but it will be task based. # of containers will not impact you.

[ayozemr]

Many thanks for the info! :-)

Do you know if they can be filtered anyway? Just to have dashboard clean

[irabinovitch]

@ayozemr The fargate integration doesn't yet support whitelisting / blacklisting containers via DD_AC_EXCLUDE. But we do intended to add that feature soon. We'll keep this issue open to track the feature request but we also have a similar internal ticket that I've added you to as a requester for follow up once its released.

[pygillier]

@irabinovitch any news on this ? Having a quite large ECS cluster leads to having a lot of theses "useless" containers showing in container view & cluttering it.

[ayozemr]

Yes, it is very messy. Especially in the honeycomb view

[vboulineau]

Hello, it has been a long (too long) wait, but we're implementing this feature, it'll be available in Agent 6/7.22

[jdmarquezf]

Hello, how are you? I have the same problem and I want to know if the charge for the containers is still being made, because I would like to stop monitoring the containers internal ecs pause