Client-go: Proposal: provide a SetLog() func

Created on 18 Oct 2016 · 34Comments · Source: kubernetes/client-go

Is it possible to provide a SetLog() in the packages? For example, I'm using 3 cache.NewInformer to watch for changes on 3 different resources, but if, for whatever reason, the connection between the client and the apiserver doesn't exist, I have this error messages in my log:

ERROR: logging before flag.Parse: E1018 14:16:07.808569    5156 reflector.go:214] k8s.io/client-go/1.5/tools/cache/reflector.go:109: Failed to list *v1.Endpoints: Get http://192.168.33.11:8080/api/v1/endpoints?resourceVersion=0: dial tcp 192.168.33.11:8080: getsockopt: connection refused
ERROR: logging before flag.Parse: E1018 14:16:08.809443    5156 reflector.go:214] k8s.io/client-go/1.5/tools/cache/reflector.go:109: Failed to list *v1beta1.NetworkPolicy: Get http://192.168.33.11:8080/apis/extensions/v1beta1/networkpolicies?resourceVersion=0: dial tcp 192.168.33.11:8080: getsockopt: connection refused
ERROR: logging before flag.Parse: E1018 14:16:08.809458    5156 reflector.go:214] k8s.io/client-go/1.5/tools/cache/reflector.go:109: Failed to list *v1.Service: Get http://192.168.33.11:8080/api/v1/services?resourceVersion=0: dial tcp 192.168.33.11:8080: getsockopt: connection refused
ERROR: logging before flag.Parse: E1018 14:16:08.809526    5156 reflector.go:214] k8s.io/client-go/1.5/tools/cache/reflector.go:109: Failed to list *v1.Endpoints: Get http://192.168.33.11:8080/api/v1/endpoints?resourceVersion=0: dial tcp 192.168.33.11:8080: getsockopt: connection refused
ERROR: logging before flag.Parse: E1018 14:16:09.811722    5156 reflector.go:214] k8s.io/client-go/1.5/tools/cache/reflector.go:109: Failed to list *v1beta1.NetworkPolicy: Get http://192.168.33.11:8080/apis/extensions/v1beta1/networkpolicies?resourceVersion=0: dial tcp 192.168.33.11:8080: getsockopt: connection refused
ERROR: logging before flag.Parse: E1018 14:16:09.811733    5156 reflector.go:214] k8s.io/client-go/1.5/tools/cache/reflector.go:109: Failed to list *v1.Endpoints: Get http://192.168.33.11:8080/api/v1/endpoints?resourceVersion=0: dial tcp 192.168.33.11:8080: getsockopt: connection refused
ERROR: logging before flag.Parse: E1018 14:16:09.811733    5156 reflector.go:214] k8s.io/client-go/1.5/tools/cache/reflector.go:109: Failed to list *v1.Service: Get http://192.168.33.11:8080/api/v1/services?resourceVersion=0: dial tcp 192.168.33.11:8080: getsockopt: connection refused

I would like to at least change the layout to be the same as my logs or/and silent the log messages until I get a new kubernetes connection.

kinfeature

Source

aanm

👍12

Most helpful comment

If we undertake the effort to switch logging frameworks, (It's not just going to be client-go), we need to have

flag compatibility - there are numerous "real" deployments using these. Make the optional to register.
dynamic package log level changes
logging formatter pluggability (one size does NOT fit all)
logging handler pluggability (different sinks exist and have value)
logging filter pluggability (dynamic and per project filtering needs exist)
glog compatibility desired.

@lavalamp @eparis @bparees you've also dealt with our logging shortcomings, other things to add to the list?

I'm disinclined to undertake the effort of a replacement that didn't address those requirements.

deads2k on 15 Jan 2018

👍4

All 34 comments

Would love to have a proper log interface & be able to provide an impl that the client could use. glog is a bit invasive here.

jimmidyson on 18 Oct 2016

👍4

Thank you for this k8s client. It's very helpful.

Do you have any advice around suppressing glog's hoisting of its command line flags into the help output of tools that use the k8s client.

For example, those flags whose descriptions are not decorated with "MyApp" are coming from glog's init function: https://github.com/golang/glog/blob/master/glog.go#L398

Usage of build/dist/darwin/myapp:
  -alsologtostderr
        log to standard error as well as files
  -config string
        MyApp: Config file that points to the K8S master. (default "./config")
  -log_backtrace_at value
        when logging hits line file:N, emit a stack trace
  -log_dir string
        If non-empty, write log files in this directory
  -logtostderr
        log to standard error instead of files
  -reload
        MyApp: Send the initial system state read from K8S to the load balancer. (default true)
  -stderrthreshold value
        logs at or above this threshold go to stderr
  -v value
        log level for V logs
  -vmodule value
        comma-separated list of pattern=N settings for file-filtered logging

ae6rt on 10 May 2017

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

fejta-bot on 24 Dec 2017

/remove-lifecycle stale

DavidHuie on 2 Jan 2018

I would also appreciate having this feature. We're currently building tooling around client-go without using glog, so we would love to have the ability to override it.

DavidHuie on 2 Jan 2018

👍1

If we undertake the effort to switch logging frameworks, (It's not just going to be client-go), we need to have

flag compatibility - there are numerous "real" deployments using these. Make the optional to register.
dynamic package log level changes
logging formatter pluggability (one size does NOT fit all)
logging handler pluggability (different sinks exist and have value)
logging filter pluggability (dynamic and per project filtering needs exist)
glog compatibility desired.

@lavalamp @eparis @bparees you've also dealt with our logging shortcomings, other things to add to the list?

I'm disinclined to undertake the effort of a replacement that didn't address those requirements.

deads2k on 15 Jan 2018

👍4

@deads2k that covers my wishlist, anyway. I suppose rate limiting or de-duping might be nice options as well, but not must haves.

bparees on 15 Jan 2018

@deads2k that covers my wishlist, anyway. I suppose rate limiting or de-duping might be nice options as well, but not must haves.

I agree those are nice to have. I think that given pluggable filters and handlers it is possible to build de-duping and rate limiting. Given pieces to build them, I won't block progress on not having them.

deads2k on 15 Jan 2018

logging formatter pluggability (one size does NOT fit all)

If you're wanting structured logs, then you really have to replace all usage of glog (even from our dependencies) in order to be useful ("almost all JSON" log output doesn't really do much good)

logging handler pluggability (different sinks exist and have value)

same comment about logger usage by dependencies

liggitt on 15 Jan 2018

logging formatter pluggability (one size does NOT fit all)
If you're wanting structured logs, then you really have to replace all usage of glog (even from our dependencies) in order to be useful ("almost all JSON" log output doesn't really do much good)

logging handler pluggability (different sinks exist and have value)
same comment about logger usage by dependencies

The same sort of argument would apply to fmt and to capnslog and whatever else other libraries do. Formatting of most messages does help significantly (ie: I get <package>.<file> in 90% of messages) . The same applies for handlers (I get a well present systemd log), getting 90% working really helps.

Mixing log4j and jsr47 doesn't work perfectly, but you don't have to have perfect coherence for big gains.

deads2k on 15 Jan 2018

same comment about logger usage by dependencies

In our projects, we have more that 80 services with countless of dependencies and so far we had no problem with structured logging (except from k8s/client-go). I found that usually dependencies just use the log.Logger interface for logging.

Could you make a list of dependencies that use glog?

tyranteon on 22 Jan 2018

Sadly it does seem reasonable to want glog to not be mandatory to use the client. This would be a fairly large effort, though. We'd have to define a new logging interface & provide a glog implementation which gives zero difference from today. (glog's interface is not easy (or possible, even?) to reimplement due to the .V(). pattern--I consider it a go language defect :( )

If we had an interface and a 100% compatible glog implementation, we could slowly roll changes out. But it's a lot of effort.

lavalamp on 22 Jan 2018

Technically you can reimplement github.com/golang/glog using another library, place the wrapper in your vendor/ dir and make e.g. golang/dep to point to your wrapper:

[[constraint]]
  name = "github.com/golang/glog"
  source = "https://github.com/yourorg/glog-wrapper"

This way you won't hit the "go language defect" @lavalamp is referring to and client-go will use your logging library without any code change.

sttts on 23 Jan 2018

❤1

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 23 Apr 2018

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

fejta-bot on 23 May 2018

/remove-lifecycle rotten

aanm on 23 May 2018

/remove-lifecycle stale

aanm on 23 May 2018

does anyone know a workaround for this? I'd like to set the format for glog logs, or disable it all together.

fiunchinho on 28 Aug 2018

@fiunchinho we used vendoring to override the provided glog with our own. See https://github.com/kubermatic/glog-gokit

tyranteon on 28 Aug 2018

👍1

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 26 Nov 2018

/remove-lifecycle stale

aanm on 26 Nov 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 24 Feb 2019

/remove-lifecycle stale

aanm on 25 Feb 2019

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 26 May 2019

/remove-lifecycle stale

DingGGu on 29 May 2019

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 27 Aug 2019

/remove-lifecycle stale

aanm on 2 Sep 2019

Since client-go now uses klog, doesn't klog.SetLogger() fulfill this proposal?

https://github.com/kubernetes/klog/blob/8422fac62d1e961e89426ffb5ae3a07f2d0bcca2/klog.go#L776

coryrc on 20 Nov 2019

👍1

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot on 18 Feb 2020

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

fejta-bot on 19 Mar 2020

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

fejta-bot on 18 Apr 2020

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

k8s-ci-robot on 18 Apr 2020

Since client-go now uses klog, doesn't klog.SetLogger() fulfill this proposal?

https://github.com/kubernetes/klog/blob/8422fac62d1e961e89426ffb5ae3a07f2d0bcca2/klog.go#L776

To be able to use klog.SetLogger() client-go needs to start using k8s.io/klog/v2