A line of output of kubectl logs for a conduit-proxy container looks like this:
1513399875.907490 TRCE tokio_io::framed_read ("serve", V4(127.0.0.1:4140)), attempting to decode a frame
The output of kubectl logs po/kube-dns-86f6f55dd5-xzgtn dnsmasq -n kube-system looks like this:
I1216 00:41:20.452866 1 nanny.go:108] dnsmasq[24]: using nameserver 127.0.0.1#10053 for domain ip6.arpa
kubedns's log is in the same format.
Notice in particular that conduit-proxy's logs's timestamps are unreadable, whereas Kuberenetes's logs's timestamps are readable. The proxy should format its timestamps like Kuberenetes does, or at least in some other more readable format.
Note that the 1216 part of the Kubernetes output appears to be the date in MMDD format, i.e. December 16th. That doesn't seem very readable either. Also the I at the beginning of the line in the Kubernetes output stands for "info"; it also uses W for "warn", etc. I'm not sure that either of these are the best conventions to follow.
Note also that kubectl logs has a separate control for timestamping --timestamps=true. If that flag isn't set then should the log just omit the timestamp? Do we need to format the log output in a particular way for Kubernetes to recognize the timestamp? If Kubernetes timestamps each line itself then we should consider removing our own timestamp both to reduce redundancy and to improve performance, as getting the system time isn't always cheap.
I'm happy to work on this, but I think there are some design decisions I'd like input on.
The timestamping that's there was done to debug subsecond timing issues and isn't needed any longer.
If fluentd is capable of tracking timestamps for us, i'd opt to just remove timestamps entirely.
Most helpful comment
Note also that
kubectl logshas a separate control for timestamping--timestamps=true. If that flag isn't set then should the log just omit the timestamp? Do we need to format the log output in a particular way for Kubernetes to recognize the timestamp? If Kubernetes timestamps each line itself then we should consider removing our own timestamp both to reduce redundancy and to improve performance, as getting the system time isn't always cheap.