Containers-roadmap: [EKS/Fargate] Automatic logging configuration for pods on Fargate

[tabern]

Fargate does not support access to writing and reading log files from the root (host) filesystem. This may require you to change you application and log agent configuration to write and read from the shared, ephemeral pod storage.

This feature will automatically configure logging for pods running on Fargate.

[lilley2412]

@tabern I know it may be too early to ask this ... is the plan to enable cloudwatch insights on each fargate node?

[dapseen]

Any idea to check logs apart from cloudwatch

[smailc]

How do we use CloudWatch Container Insights with a Fargate only EKS cluster?

[casret]

@smailc would love to get an answer to this or other workarounds people are using. For statsd as well.

[dalgibbard]

@smailc would love to get an answer to this or other workarounds people are using. For statsd as well.

When we were reviewing this, we wanted to export our logs to datadog specifically; the workaround would have been to:

• Amend the application's configuration to log to file (ephemeral container disk)
• Add a fluentbit sidecar to poll the logfile and ship to datadog (or other supported service as desired).

This however, still did not fully meet our compliance for centralised logging, because errors that occur in the container/application which don't trigger the log handler (maybe a stack trace or other unhandled exception), would not be exported.

One possibility would be to write your own application wrapper which exports logs received on stdin/stdout perhaps, but again, non-standard and laborious. There's still the potential case where an application or healthcheck state fails, and the container is restarted before the log export process has parsed the logs too (perhaps).

[tabern]

solved as part of #701

[mattFN2187]

Looks like Solarwinds just launched a thing called "rKubelog" ... open source connector claiming to be a simple way to send logs from a K8s cluster to their Papertrail product without having to configure DaemonSets, sidecars, fluentd, or persistent claims: https://documentation.solarwinds.com/en/Success_Center/papertrail/Content/kb/configuration/rkubelog.htm?cshid=ptm-rkubelog

public github repo: https://github.com/solarwinds/rkubelog
product blog: https://www.papertrail.com/blog/quick-and-easy-way-to-implement-kubernetes-logging/

If anyone tries it, let me know what you think. Curious to see how it works for you.

[tabern]

Merging this with #701 as they are the same issue.

[GeoffWilliams]

If anyone tries it, let me know what you think. Curious to see how it works for you.
@mattFN2187 I've been trying it out for the last week. Works as advertised, very happy and simple to setup

[akshayram-wolverine]

Now live: https://aws.amazon.com/about-aws/whats-new/2020/12/amazon-eks-adds-built-in-logging-support-for-aws-fargate/