Loguru: KeyError while using on distributed environments

[talz]

@ https://github.com/Delgan/loguru/blob/master/loguru/_logger.py#L1418
_make_log_function::log_function tries to access frame.f_globals["__name__"]

In some scenarios (such as when running from a dask worker - assuming because of an rpc simulated stack frame) __name__ is not defined, and a KeyError exception is thrown.

Maybe setting a safe dict.get call with default value would suffice to fix this issue.

[Delgan]

Thanks for reporting this!

So, you are saying this is easily reproducible by using loguru inside a dask worker? I'm not familiar with this library, but I would like to investigate how it handles Python frames internally. Otherwise, .get() should be an easy fix indeed.

[Delgan]

@talz @giedroyc Could you please show me some code snippet that would trigger the error please? I actually do not know how to reproduce this.
The entire traceback and the output of pip freeze (so I know which dask version is used) could help too.

Thanks!

[Delgan]

I was not able to reproduce it but I fixed it by fall-backing to None if a KeyError is raised while accessing f_globals["__name__"]. This means that in such case, record["name"] will be equal to None for logs coming from the Dask worker.

It should be noted that the name can be manually overridden using the patch() method:

logger = logger.patch(lambda record: record.update(name="my_module"))

[radugrosu]

I get the same problem with the latest version of loguru. You can reproduce it using this code (you need to install ray with pip install ray first):

from loguru import logger
import ray
ray.init(ignore_reinit_error=True)

@ray.remote
def f(arg):
    logger.info("arg is {}", arg)

ray.get([f.remote(i) for i in range(2)])

The exact error is:

File "/home/radu/anaconda/envs/py36/lib/python3.6/site-packages/loguru/_logger.py", line 1407, in log_function
    name = frame.f_globals["__name__"]
KeyError: '__name__'

[Delgan]

Hey @radugrosu, what is the output of print(loguru.__version__)?

Although you think you're using the most recent version, the exception you reported definitely looks like it comes from Loguru v0.2.5 (latest version is v0.3.2).

If you look at line 1407 of 0.2.5, it indeed contains f_globals["__name__"]: https://github.com/Delgan/loguru/blob/ae123950db11e2cf6abb0d3b558d6b8768a6694e/loguru/_logger.py#L1407

While for the latest version, it moved to line 1555: https://github.com/Delgan/loguru/blob/d620a117777bb0cc5b3825e0eaf862601e9ab7b8/loguru/_logger.py#L1555

You can try to update Loguru with python -m pip install --update loguru, otherwise maybe you have two versions installed, and Python fails to retrieve the most recent one? :confused:

[radugrosu]

Sorry @Delgan , you are right - I had upgraded, but forgot forgot somehow to restart my ipython kernel :blush: . Thank you for the quick response and for having fixed the issue. Oh, and for loguru, generally :1st_place_medal: .