Containers-roadmap: Enable support for custom log drivers

[dangerousplay]

Summary

Hello, our company needs to use a custom log driver designed to meet our requirements, but unfortunately, the ECS agent does not support custom log drivers.

Description

Because Docker's built-in gelf log driver does not meet all of our company's requirements, we have developed a plugin for Docker, a custom log driver that transforms and handles the logs of applications running on Docker. One of the transformations is the aggregation of n partial logs into a single log containing the entire message. We also have resolved a known TTL DNS issue that affects the log driver gelf (Reported at issue https://github.com/moby/moby/issues/17904#issuecomment-194403940). Our solution is working, but when we tried to use the log driver on the ECS agent, we figure out that it does not support custom log drivers.

Maybe something should be done in this session code:

https://github.com/aws/amazon-ecs-agent/blob/f95f731bce74e6276f9fd8bfbb87a62d3a790ce8/agent/config/config.go#L295-L304

[shubham2892]

Hi @dangerousplay, unfortunately we currently do not support custom log drivers.

I will mark this as a feature request.

[hermes-pimentel]

Hi @dangerousplay, unfortunately we currently do not support custom log drivers.

I will mark this as a feature request.

Sumoligic is a custom log driver and is supported...

[petderek]

I think a more specific way to say this is that we don’t currently support arbitrary custom logging drivers. There’s implications on the service side with regard to capabilities and placement.

[fitzoh]

There is a workaround for people using EC2 based ECS.

If you set your custom driver as the default for the docker daemon (docs) and don't provide any configuration through ECS, the custom driver will be used (discovered/verified this while trying to test out loki)