Prophet: Feature Request: Matplotlib should be an optional dependency

[joincamp]

A little background might help explain why this would be a nice feature.

From the Prophet whitepaper:

It is worth noting that we are ignoring the typical considerations that scale
implies: computation and storage. We have found the computational and infrastructure
problems of forecasting a large number of time series to be relatively
straightforward – typically these fitting procedures parallelize quite easily and
forecasts are not difficult to store in relational databases.

(emphasis mine)

Which totally makes sense, and in my very narrow, but arguably common use-case, I'm on AWS and Lambda is the clear path for me. Lambda is amazing for scaling little things up and getting a lot of compute done without worrying so much about things like having to pay for servers when you aren't using them.

So here is the rub... The dependency chain in this project is like an iceberg. And I get it, but many of the libraries just aren't even used in this type of use case, where you are just dealing with the dataframes and you don't need the plots. The base package size for all the dependencies is >700mb. In order to run on lambda I have to get that down to 250mb uncompressed, and I have been able to do this by monkey patching pystan and fbprophet. I'd rather not have to maintain a separate fork(of this project at least) and would love if having matplotlib as an optional dependency could make it into the mainline. I have no idea if there is an elegant way of having a smaller pystan, but I'll take on that burden).

If I were to find a good way of doing this, would you consider merging it in to master?

[joincamp]

I have also filed an issue with stan-dev/math that should help out with overall distributable size if they are amenable. https://github.com/stan-dev/math/issues/545

[joincamp]

Based on https://github.com/stan-dev/math/issues/545#issuecomment-299038368, it looks like after the stan models have been pickled, we can actually create a deployable package with a lot of the bulk deleted.

[bletham]

I'm amenable to making matplotlib optional for those that do not need the plotting. I would though want to be sure that when users install with pip it does install/upgrade matplotlib by default. It could be confusing for someone to install Prophet and then not understand why the plotting isn't working.

[bletham]

If someone would like to make a PR for this, we can just put the matplotlib import in a try/except and disable the plotting methods if it did not import. matplotlib should remain a dependency in pip though, so it will be installed by default with a normal pip install.

[jpcaruana]

or maybe we could make another package, say fbprophet-plot, for those who want to plot stuff

[bletham]

@jpcaruana Thanks for the PRs. I don't think there is a way to empirically verify this, but I suspect that the vast majority of fbprophet users will be using this without such tight space constraints and will want to use the plotting. More broadly, we also feel that visualization is an important part of the forecasting process and as such the plotting functionality is a core part of the package.

I appreciate the value in making it possible to run Prophet in space-constrained environments where the matplotlib install is not possible. However, this needs to be done without making it more difficult for mainstream use of fbprophet. Prophet-without-plotting needs to be a special case and not the default. In particular, pip install fbprophet needs to install matplotlib and have plotting work.

[jpcaruana]

It is very hard to make it work under MacOS X + pyenv (so hard I couldn't make it happen, even after trying for hours).

The import matplotlib is just killing me and I don't need it at all. This is a very special case, but please consider my second PR where the import matplotlib statements reside whitin plot functions.

[ajbouh]

How can we get an(y) implementation of this into a release? :)

[bletham]

If this satisfies the use-case:

try:
    from matplotlib import pyplot as plt
    from matplotlib.dates import MonthLocator, num2date
    from matplotlib.ticker import FuncFormatter
except ImportError:
    logger.error('Unable to import matplotlib. Plotting will not work.')

without changing requirements.txt or install_requires, then I can commit it right now and it'll go in the next release.

[bletham]

OK, https://github.com/facebook/prophet/commit/7f9e4b80c1b082fd221d6e189c3938ff31ce74c1 does two things:
1) Puts the importing of matplotlib in a try/catch as above, so that everything that doesn't use plotting will still work on a system without matplotlib
2) Reads install_requires from requirements.txt, so you can remove dependencies by altering requirements.txt

The workflow for using prophet without matplotlib would be:
1) Download the source
2) Remove matplotlib from requirements.txt (easily done with a script)
3) python setup.py install will install fbprophet without installing matplotlib, and all non-plotting functionality will work.

Does that work for everyone?

[bletham]

You can download the v0.3 branch to try it out.

[ajbouh]

I suppose I can fork fbprophet and just need to edit/resolve conflicts in requirements.txt. This is much better than needing to deal with conflicts in source files.

A solution that worked with pypi would be ideal, but this moves things forward for now. Thanks!

[bletham]

In v0.3 you can now load prophet without matplotlib installed and it will just print a warning (not fail with ImportError). It is still listed as a install dependency in PyPI, but you can get around that by installing from source (python setup.py install) and removing matplotlib from the requirements.txt. I think that should satisfy most of the need for this, but we can reopen if there is more to be done here.

[wwguo]

Is it possible to change logger.error('Unable to import matplotlib. Plotting will not work.') to a warning? If someone don't need to plot anything, it's actually not an error.

[bletham]

@wwguo Is the issue that you want to suppress this message without suppressing other messages logged at the error level? Or just the general messaging?

I do see it as being closer to error than warning since for a typical user the plotting is a core functionality which will not work with this ImportError.

FYI right now there is nothing else logged at either WARNING or ERROR levels, so setting the logging level to suppress ERROR would have the same effect as if it were changed to WARNING level and suppressed there. Both of these would of course suppress all of the INFO messages, so it'd probably be better just to use a logging filter. (Which would work independently of the logging level).

[wwguo]

I ask the question because my colleague came to me and told me that I got an error there. I hacked the forecaster.py file, so it's not a problem to me anymore. Thanks for replying.

[bletham]

I see. It prints an error, but does not raise actually raise an exception, so everything will continue to work and no modification should be needed.