Prophet: Package is overriding log output

[collincburton]

When the fbprophet module is loaded, my log entries don't write to my designated log file. Info level logs print to the console. If you comment out the import Prophet line, this writes to a log as expected.

import logging
from fbprophet import Prophet


logging.basicConfig(filename='log_test.log', level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

logging.debug("test debug message")
logging.info("Info message")

[collincburton]

This is my first attempt at a bug fix on a package, but I'm going to take a try on it.

[bletham]

Thanks! As an FYI, we are about to push out the v0.3 branch, so you'd want to do any investigating there and not on master.

[collincburton]

I think I found the issue. It looks like prophet is importing the logging package and using it to raise exceptions, warnings, info statements, etc. We can replace those with the equivalent functions in base python and I think that should fix it. It's been a great learning experience so far.

[bletham]

The logging package is the standard way to log in Python, so I'd be very hesitant to not use it. I think the right thing to do here is determine if there is something in fbprophet that needs to be done to enbale changing the default logging config like in your example.

[collincburton]

I get that the logging package is the standard way to log, but is it standard practice to put it in the package itself as opposed to letting the user import logging and create their own logs as they want to? If a user has started their log and the fbprophet package raises an exception, warning, etc., the user can record the event as they see fit as opposed to how it is defined within fbprophet. I'm not sure what best practice is in this matter and I'll do a little more research.

[Teoretic6]

Found this issue too today, although in different form.

I set up logging only into file, but when I import Prophet, logging information also starts to log into terminal, in which I'm rinning my code.

Sample code
If you comment out Prophet import in code below, debug lines 'log this' and 'log that' stop to log into terminal and start to log only to log file as intended.

import logging
from fbprophet import Prophet

if __name__ == '__main__':
    # Setup logging to file
    root_logger = logging.getLogger()
    root_logger.setLevel(logging.DEBUG)
    handler = logging.FileHandler('logfile.log',
                                  'w',
                                  'utf-8')
    handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
    root_logger.addHandler(handler)
    logging.debug('log this')
    logging.debug('log that')

I think that there is something with logging setup in Prophet, can investigate further.

Furthermore pandas also has logging functionality (when loading dataframe to BigQuery for example), but importing pandas doesn't make logging also log to terminal (as far as I tried)

[collincburton]

I've been digging around in this. I think I've figured out the logging bugs in prophet, but it looks like they extend into the pystan package.

[collincburton]

This code will generate the same issue as my original code, but the only difference is that I'm importing pystan instead of fbprophet.

import logging
import pystan


logging.basicConfig(filename='log_test.log', level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

logging.debug("test debug message")
logging.info("Info message")
logging.warning("Warning message")

[collincburton]

I have a PR submitted to pystan here https://github.com/stan-dev/pystan/pull/466

[bletham]

Following along in the Pystan issue, I've decided to handle the issue the same way they have (https://github.com/facebook/prophet/commit/f44285038cafe34611c9fa95ad47dd54f9d91119).

This means that logging settings can be adjusted the same way they are adjusted in pystan, just substitute fbprophet for the name of the logger: https://github.com/stan-dev/pystan/blob/develop/doc/logging.rst

[bletham]

Pushed to PyPI.