Operator-sdk: Changing log level

[mossuchida]

Please let me know how to change the log level. Currently, all the logs are displayed. I tried the following, but it complained: syntax error: non-declaration statement outside function body
Error: failed to build operator binary

atom := logf.Log.NewAtomicLevel()
atom.SetLevel(logf.Log.ErrorLevel)
var log = logf.Log.WithName("controller_moss")

[mossuchida]

I see this doc for --zap-level, but it talks about command line command against containers & I cannot find any example on how to use it:

https://github.com/operator-framework/operator-sdk/blob/master/doc/user/logging.md

[joelanford]

@mossuchida The main.go that gets scaffolded sets up the zap flagset, and the sets up a logger that is configured by those flags.

If you didn't change these lines in your main.go, you can pass --zap-level to set the log level. If you're using your own logr implementation, you'll have to setup the flags and configure the log level yourself.

[mossuchida]

I am newbie to Go lang (I am using it for 3 ~ 4 weeks), I am trying differently, but so far I am not successful. Would you provide me a code sample? I am using default logger.

[joelanford]

@mossuchida Did you use operator-sdk new to create a project? If so, you should have a file in your project at cmd/manager/main.go that looks like the file from the links I posted. With no changes to that file, you should be able to run operator-sdk up local --operator-flags='--zap-level=10' and see much more verbose logging.

Here's a much simpler snippet with just the pieces important for logging:

package main

import (
    "github.com/operator-framework/operator-sdk/pkg/log/zap"
    "github.com/spf13/pflag"
    logf "sigs.k8s.io/controller-runtime/pkg/runtime/log"
)

var globalLog = logf.Log.WithName("global")

func main() {
    // Add the zap logger flag set to the CLI. The flag set must
    // be added before calling pflag.Parse().
    pflag.CommandLine.AddFlagSet(zap.FlagSet())

    pflag.Parse()

    // Use a zap logr.Logger implementation. If none of the zap
    // flags are configured (or if the zap flag set is not being
    // used), this defaults to a production zap logger.
    //
    // The logger instantiated here can be changed to any logger
    // implementing the logr.Logger interface. This logger will
    // be propagated through the whole operator, generating
    // uniform and structured logs.
    logf.SetLogger(zap.Logger())

    scopedLog := logf.Log.WithName("scoped")

    globalLog.Info("Printing at INFO level")
    globalLog.V(1).Info("Printing at DEBUG level")
    scopedLog.Info("Printing at INFO level")
    scopedLog.V(1).Info("Printing at DEBUG level")
}

Default log level:

$ go run main.go
{"level":"info","ts":1559866292.307987,"logger":"global","msg":"Printing at INFO level"}
{"level":"info","ts":1559866292.308039,"logger":"scoped","msg":"Printing at INFO level"}

Debug log level:

$ go run main.go --zap-level=1
{"level":"info","ts":1559866310.065048,"logger":"global","msg":"Printing at INFO level"}
{"level":"debug","ts":1559866310.0650969,"logger":"global","msg":"Printing at DEBUG level"}
{"level":"info","ts":1559866310.065119,"logger":"scoped","msg":"Printing at INFO level"}
{"level":"debug","ts":1559866310.065123,"logger":"scoped","msg":"Printing at DEBUG level"}

[lilic]

@joelanford Should we add this to the examples in the logging? I got this question at least twice before, so might make sense. :)

[mossuchida]

I am running in kubernetes env (IKS), I am assuming it is using Dockerfile in build dir & want to see less logs in the deployed env (not in my local), but this is what I see. Would you let me know What to change? I just want to change it to print error case only in deployed pod logs using "kubectl logs pod-name" (I added some info logs, but I don't want to see them)

Dockerfile has these lines to startup operator SDK, but I don't have user_setup file locally (it seems like it comes with the image). How do you pass --operator-flags='--zap-level=10' in Dockerfile?

RUN /usr/local/bin/user_setup
ENTRYPOINT ["/usr/local/bin/entrypoint"]

[joelanford]

@mossuchida Rather than changing the docker image via the Dockerfile, I'd suggest that you set --zap-level in your operator deployment (deploy/operator.yaml), like this:

args:
- '--zap-level=error'

This args field should be in the container spec for your operator. Here's the full example based on a basic operator.yaml scaffold:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: memcached-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      name: memcached-operator
  template:
    metadata:
      labels:
        name: memcached-operator
    spec:
      serviceAccountName: memcached-operator
      containers:
        - name: memcached-operator
          # Replace this with the built image name
          image: REPLACE_IMAGE
          command:
          - memcached-operator
          args:
          - '--zap-level=error'
          imagePullPolicy: Always
          env:
            - name: WATCH_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: OPERATOR_NAME
              value: "memcached-operator"

[joelanford]

@LiliC I created PR #1532

[mossuchida]

Thanks, --zap-level=error worked. Now I am changing the log from reqLogger.Info to reqLogger.Debug, but it complains that it doesn't exist. I tried to use reqLogger.V(1).Info with "--zap-level=info", but it prints out reqLogger.V(1).Info also. Would you provide complete --zap-level options & reqLogger options that I can use for operator.yaml & controller.go?

[joelanford]

@mossuchida the logging doc describes the --zap-level flag as:

--zap-level string or integer - Sets the zap log level (debug, info, error, or an integer value greater than 0). If 4 or greater the verbosity of client-go will be set to this level.

So you can use --zap-level=error, --zap-level=info, --zap-level=debug, --zap-level=1, --zap-level=2, --zap-level=3, etc. Note that debug and 1 are equivalent and increasing numbers mean more verbosity.

These levels correspond to the logr interface as follows:

• --zap-level=error - you'll see reqLogger.Error() logs only.
• --zap-level=info - you'll see everything above and reqLogger.Info(), and reqLogger.V(0).Info() logs.
• --zap-level=debug (and --zap-level=1) - you'll see everything above and reqLogger.V(1).Info() logs.
• --zap-level=2 - you'll see everything above and reqLogger.V(2).Info() logs.

Also to address your questions:

Now I am changing the log from reqLogger.Info to reqLogger.Debug, but it complains that it doesn't exist.

Correct, see the logr interface to see what functions are available on the reqLogger.

I tried to use reqLogger.V(1).Info with "--zap-level=info", but it prints out reqLogger.V(1).Info also.

That shouldn't be happening. Is it possible that you are overriding the log level somewhere? If so, can you reproduce it based on the simple snippet code from my earlier comment?

[mossuchida]

Thanks, it worked! It looks like I was not logged into docker & image push was denied, there were no V(1) in the code.