Amazon-ecs-agent: Log configuration: Support `none` as a valid logging driver option

[Schweigi]

Despite setting the log configuration of a Task Definition to <none> (default) it seems that the container logs are still available with docker logs. I would expect that in this case no logs should be available and written to the disk of the ECS cluster instance.

According to the Docker documentation if no logging parameters are specified then json-file logging is used. I guess the problem happens because the ECS agent won't pass the none logging driver to Docker and Docker falls back to json-file.

Is it possible to pass none as logging drive when <none> is selected in the Task Definition or offer a real None option in order to not write any logs to the ECS cluster instance disk?

[aaithal]

@Schweigi Thanks for reporting this.

ECS currently does not support the none logging driver. The list of supported logging drivers can be found here.

From the console dropdown, if you don't select a logging driver, a null value is assigned for the logConfiguration field in the task definition ("logConfiguration": null), which is probably why logging hasn't been disabled for the container.

Could you please provide us more details about your use-case for ECS supporting none logging configuration so that we can prioritize this as a feature request?

Thanks,
Anirudh

[Schweigi]

We came across this issue by mistake because one of our dockerized services logged over 500MB/day to the console. This filled the ECS cluster node disk as Docker was dutifully writing those logs to the disk for the container log. We fixed it now by disabling the console log in production mode (which was anyway a good thing).

In general it would be nice to have an option to disable Docker container logging to prevent such issues from happening. Because currently if there is no log driver selected (as it is by default) then there won't be any limits set (e.g. max-log-size) which means that the Docker container logs grow until the ECS cluster node disk is full.

[aaithal]

@Schweigi Thanks for providing those details. I'll log this as a feature request. I'll also modify the title to reflect the same.

Thanks,
Anirudh

[hltbra]

I didn't know that ECS didn't support "none" as a valid log driver. I noticed that I can successfully create a task definition with none, but it seems that the ecs-agent never picks up tasks to run.

What's the official workaround, @aaithal? I am thinking about using json-file and setting a max-size.

My situation: We are spending too much with CloudWatch Logs and some of our applications have other ways to monitor their health and statuses, their logs aren't that important at this point of the project life. I'd like to give my developers the ability to enable/disable logs for their ECS containers any time (we have an internal tool on top of ECS using cloudformation to manage task defs/services/clusters).

[aaithal]

Hi @hltbra,

Apologies for the delayed response here. Although "none" is recognized as a logging driver by ECS, our console doesn't yet accept it as one of the parameters. I have raised this issue internally with the console team and will update this issue when it's fixed.

You can register a task definition using the AWS CLI with "none" logging driver. I was able to register the following task definition using the CLI:

$ cat sleep.json
{
        "family": "sleep",
        "containerDefinitions": [
            {
                "name": "essential_sleep",
                "image": "ubuntu:latest",
                "cpu": 25,
                "entryPoint": [
                    "bash",
                    "-c"
                ],
                "memory": 100,
                "command": [
                    "while true; do sleep 5; done"
                ],
                "essential": true,
                "logConfiguration": {
                    "logDriver": "none"
                }
            }
        ]
}

However, I also found that ECS Agent needs a code change to recognize the "none" logging driver while testing this out. I've submitted PR #1041 to fix that, which should be going out with the next ECS Agent release.

Thanks,
Anirudh

[samuelkarp]

We have released ECS agent v1.15.0, which adds this feature.

[gomlgs]

After this release, my task definitions seem to require the attribute 'com.amazonaws.ecs.capability.logging-driver.none', and there is no direct way in the console to remove that requirement. I have to now edit the JSON and remove the logConfiguration section.

Will this be fixed? This issue is fairly non-obvious, and new revisions of my tasks no longer schedule because the containers don't have that attribute.

[aaithal]

Hi @gomlgs, this seems to be an issue with our task definition registration page. We're working on a fix for this and will let you know when we have more information.

Thanks,
Anirudh

[aaithal]

Hello everyone,

The issue with the task definition registration page should be fixed now. Please let us know if you keep running into this.

Thanks,
Anirudh

cc @gomlgs

[Tiny-wlx]

Hi @aaithal, when configure logdirve from cloudformation,
"LogConfiguration": {
"LogDriver": null
},
it show error like:

"[/Resources/CollectorTask/Type/ContainerDefinitions/0/LogConfiguration/LogDriver] 'null' values are not allowed in templates"

Does cloudformation not support this feature?

[adnxn]

@Tiny-wlx, if I understand correctly, I think you want the following in your task def:

"logConfiguration": {
                    "logDriver": "none"
                },

[Recodify]

I still do not see the 'none' option via the task definition console.
Perhaps another work around is to use the splunk log driver but specify an invalid splunk-url and splink-token?

[fredericgermain]

Trying to use splunk log driver with forged splunk-url and splunk-token fails (tried http://localhost, ...), .

Or I haven't try hard enough but I have no idea how splunk works.