Loguru: Change level of default handler

[jetheurer]

First off, this library is terrific, I found it via the podcast Python Bytes and I've been using it ever since.

So here is my question: I understand, the default handler for from loguru import logger goes to sys.stderr.

When I try: logger.add(sys.stderr, level="INFO"), I still get DEBUG level messages in the terminal.

My goal is to change the level of the logging to sys.stderr. I don't have any other handlers.

[Delgan]

Hello @jetheurer, happy to know that you like Loguru ! 😃

So, indeed, by default a sys.stderr handler is added with an "INFO" level.

You can add as many handlers as you want, this means that if you do logger.add(sys.stderr, level="INFO") while the default handler has not been removed, your log messages will be dispatched twice to sys.stderr!

This is why you still see DEBUG messages. Those are the one from the default handler which co-exists with your INFO handler.

The solution is to first .remove() the default handler, before adding a new one which logs to stderr.

Alternatively, you can set the LOGURU_LEVEL environment variable to "INFO", so the default handler will be started with your preferred level all the times.

[jetheurer]

@Delgan Thank you! I've got

log.remove()
log.add(sys.stderr, level=config.LOG_LEVEL)

in my codebase now. I missed that the .remove() method removes the previously added handler (meaning the default one). For some reason I thought I'd have to know the default handler's id.

[Delgan]

@jetheurer If no argument is provided to logger.remove(), all existing handlers are removed.

You can also remove the default handler using logger.remove(0), because the default handler will always have the id 0.

[RichardDally]

Hello @Delgan,

I'm wondering why default logger is set to DEBUG level ?
I would like to use it directly in production by replacing existing logger but it needs tweaking (remove/add sink).
That's unfortunate :(

Anyway, it's great pieces of code you wrote, I'm keen to use it more !

Regards,

[RichardDally]

Actually I've found a smarter way to change less code: replacing logger.debug to logger.trace
Now there is no need to tweak default sink.

[Delgan]

Hi @RichardDally. :)

The rational behind setting the default level to DEBUG is as follow: every new project first goes through a development phase during which debug messages are much helpful. As a loguru user, you will spend most of your time developping your application. When your software is fully working and ready to release, you can take the time to properly add handlers and configure the level based on a command line argument for example.

So it makes sense, from my point of view, that you can start any Python application and see debug messages without any further configuration.

from loguru import logger

logger.debug("Let's go to work")

If you want to replace standard logging with Loguru, you also need to replace logger.addHandler() and others setup steps, right? So, could you not simply use logger.add(sys.stderr, level="INFO")?

[eric-tramel]

@Delgan On this topic, what happens in the case that one wants to change the logging level printed to the (deafult) logging sink sys.stderr, but _after_ other sinks may or may not be added? The question here revolves around how to locate the sys.stderr in the resulting list of handlers.

As an example:

import sys
from loguru import logger

# Add a file sink
logger.add("howdy.log") 

# Remove to reset logging level to something else
logger.remove(0)
logger.add(sys.stderr, level="WARNING")

# Add some other file sink later
logger.add("yolo.log")

# Now adjust the logging level on sys.stderr
# .... ? ....

Is there a general approach to adjusting the logging level on sys.stderr at the end of the above example, or in some other example where you can't definitively say that the sys.stderr exists at the beginning or end of the handler list?

The only think I can come up with the moment is to do some unadvisable digging and make a level reset behave more like the following:

import sys
from loguru import logger

# Add a file sink
logger.add("howdy.log") 

# ...

# Set the logging to something, abuse sys.stderr @ index 0
logger._handlers[0].levelno = logger.level("WARNING").no

# ...

# Add some other file sink later
logger.add("yolo.log")

# Adjust logging level again, abuse sys.stderr @ index 0
logger._handlers[0].levelno = logger.level("DEBUG").no

[Delgan]

Hi @eric-tramel. I heard about Okwin just a few weeks ago, what a coincidence to see you here. 😄

There is actually no way to update a handler once it has been added. It is a deliberate choice in order to keep a minimal API. There are however several possible solutions for your use case.

You can use the handler id returned by .add() to later stop it before re-adding it with a new level.

logger.add("howdy.log") 

logger.remove(0)
i = logger.add(sys.stderr, level="WARNING")

logger.add("yolo.log")

logger.remove(i)
logger.add(sys.stderr, level="DEBUG")

Alternatively, you can configure a dynamic level by using a filter function. For example, if you intend to set the level to "DEBUG" just temporarily:

def my_filter(record):
    if "warn_only" in record["extra"]:
        return record["level"].no >= logger.level("WARNING").no
    return True

logger.add(sys.stderr, level="DEBUG", filter=my_filter)

# Use this logger first, debug messages are filtered out
warn_logger = logger.bind(warn_only=True) 

Otherwise, you can use just one logger with a more advanced filter:

class MyFilter:

    def __init__(self, level):
        self.level = level

    def __call__(self, record):
        levelno = logger.level(self.level).no
        return record["level"].no >= levelno


my_filter = MyFilter("WARNING")
logger.add(sys.stderr, filter=my_filter, level=0)
logger.warning("OK")
logger.debug("Nope")
my_filter.level = "DEBUG"
logger.debug("OK")

[eric-tramel]

Hi @Delgan, many thanks to you for your detailed response! I think you could have thrown a RTFM at me for using the return of the logger.add(), as this accomplishes directly the task of identifying the sink/handler assigned to sys.stderr.

I also like the filter approach, I'll have to revisit my rushed implementation and see if this gives something more elegant.

❤️ from Owkin 🤗

[Delgan]

I'm closing this issue as I added a page to the documentation for referencing this kind of useful code snippets, for example: Changing the level of an existing handler

[yuzhujiutian]

How to make all py files take effect with the modified default logger level?

e.g.

proj_src
    main.py
    dirX
       x.py
    dirY
       y.py

add the following code to the main.py, main.py logger.level is INFO, bug x.py y.py are still DEBUG level.

from loguru import logger
from loguru._defaults import env

logger.remove(0)
LOGURU_LEVEL = env("LOGURU_LEVEL", str, "INFO")
logger.start(sys.stderr, level=LOGURU_LEVEL)

@Delgan

[Delgan]

Hey @yuzhujiutian. I have trouble understanding how this could happen... Are you calling .add() in x.py and y.py too?

[yuzhujiutian]

@Delgan

no calling .add() in x.py and y.py

How do I write logger level changes only once (e.g. main.py) and other all python files (e.g. x.py and y.py) take effect at the same time?

[Delgan]

@yuzhujiutian The logger of others files should be updated at the same time you change the level in main.py.

That's weird if you are not seeing that. Could you please share with me the content of x.py and y.py, and how you are using them?

[yuzhujiutian]

@Delgan
I'm sorry, maybe I didn't make myself clear.

I want a global configuration to set the default logger level. and how do I do?

For example, after a file executes the line of code in main.py

proj_src
    main.py
    dirX
       x_test.py
    dirY
       y_test.py


# content of main.py
from loguru import logger
from loguru._defaults import env

logger.remove(0)
LOGURU_LEVEL = env("LOGURU_LEVEL", str, "INFO")
logger.start(sys.stderr, level=LOGURU_LEVEL)

if __name__ == '__main__':
    logger.debug(f'start to Enable multi-device concurrent execution test case based on pytest-xdist')
    logger.info(f'start to test')
    pytest.main(['-n 5'])


# content of x_test.py
import inspect
from loguru import logger

class TestX:
    def test_x():
        logger.debug(inspect.stack()[1])
        logger.info(f'start to test x')
        assert 1==1

# content of y_test.py
import inspect
from loguru import logger

class TestY:
    def test_y():
        logger.debug(inspect.stack()[1])
        logger.info(f'start to test y')
        assert 1==2

the logger print operations that follow it all follow that 'INFO' level in x_test.py and y_test.py.

[Delgan]

@yuzhujiutian I see, thanks for the clarifications!

Well, this seems to happen because of pytest-xdist which run tests on muliple processes. Without the -n 5 option, it works fine. I don't know the exact internals of pytest, but it looks like changes made to the logger in main.py does not impact the logger in x_test.py and y_test.py (it's not the same object!).

You need to modify the logger object used by x_test.py and y_test.py. Are you aware of conftest.py? This is a special file that can be used by pytest to configure the tests. By creating a custom pytest.fixture() that is automatically called before running tests, I was able to circumvent your problem:

# content of project_src/conftest.py 
import sys
import pytest
from loguru import logger

@pytest.fixture(autouse=True)
def set_logger():
    logger.remove()
    logger.add(sys.stderr, level="INFO")

[yuzhujiutian]

yes, i know conftest.py
it is able to circumvent my problem

thank you very much. @Delgan

[crizCraig]

Just to be clear, I think @Delgan is recommending that folks do this via the env. Without setting the env, you are defaulted to debug mode for development. Then in production you set LOGURU_LEVEL in the environment. To change the level in development to see what prod logs look like, you can just do:

LOGURU_LEVEL=INFO python my_script.py