Pandas: BUG: epoch2num matplotlib deprecation warning

[dstansby]

• [x] I have checked that this issue has not already been reported.

• [x] I have confirmed this bug exists on the latest version of pandas.

• [x] (optional) I have confirmed this bug exists on the master branch of pandas.

---

Problem description

Using the current matplotlib master branch, pandas raises the following deprecation warning:

/Users/dstansby/miniconda3/envs/dev/lib/python3.8/site-packages/pandas/plotting/_matplotlib/converter.py:256: MatplotlibDeprecationWarning: 
The epoch2num function was deprecated in Matplotlib 3.3 and will be removed two minor releases later.

It looks like this line is present on pandas master, so should probably be changed (I'm not sure how) to avoid raising this deprecation warning.

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.8.3.final.0
python-bits : 64
OS : Darwin
OS-release : 19.4.0
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 1.0.4
numpy : 1.18.5
pytz : 2020.1
dateutil : 2.8.1
pip : 20.0.2
setuptools : 47.1.1.post20200604
Cython : None
pytest : 5.4.3
hypothesis : None
sphinx : 3.0.4
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.5.1
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.15.0
pandas_datareader: None
bs4 : 4.9.1
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : 4.5.1
matplotlib : 3.2.1.post2858+gc3bfeb9c3
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : 5.4.3
pyxlsb : None
s3fs : None
scipy : 1.4.1
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
numba : None

[igavronski]

Same here:

pip list
Package         Version
--------------- ---------
certifi         2020.6.20
chardet         3.0.4
cycler          0.10.0
idna            2.10
kiwisolver      1.2.0
matplotlib      3.3.0
numpy           1.19.0
opencv-python   4.3.0.36
pandas          1.0.5
Pillow          7.2.0
pyparsing       2.4.7
python-dateutil 2.8.1
pytz            2020.1
requests        2.24.0
six             1.15.0
urllib3         1.25.9

python -V
Python 3.8.4

Platform: Windows 10

[igavronski]

The problem is here:
https://github.com/pandas-dev/pandas/blob/master/pandas/plotting/_matplotlib/converter.py#L256

[misantroop]

Is there a workaround or how to approach this issue?

[0x26res]

Is there a workaround or how to approach this issue?

If you really need a workaround you can monkey patch pandas.plotting._converter._dt_to_float_ordinal which calls the deprecated function dates.epoch2num

Maybe there's a cleaner way, but I can't think of one.

import matplotlib.dates as dates
import numpy as np
import pandas as pd
import warnings

from pandas.core.dtypes.common import is_datetime64_ns_dtype
from pandas.core.dtypes.generic import ABCSeries
from pandas.core.index import Index


def my_dt_to_float_ordinal(dt):
    """
    Convert :mod:`datetime` to the Gregorian date as UTC float days,
    preserving hours, minutes, seconds and microseconds.  Return value
    is a :func:`float`.
    """
    if (isinstance(dt, (np.ndarray, Index, ABCSeries)
                   ) and is_datetime64_ns_dtype(dt)):
        base = dates.EPOCH_OFFSET + np.asarray(dt.asi8 / 1.0E9) / dates.SEC_PER_DAY
    else:
        base = dates.date2num(dt)
    return base


import pandas.plotting._converter

pandas.plotting._converter._dt_to_float_ordinal = my_dt_to_float_ordinal

df = pd.DataFrame(
    {
        "datetime": [pd.Timestamp("2020-01-01", tz="UTC"), pd.Timestamp("2020-01-02", tz="UTC")],
        "value_1": [1.0, 2.0],
        "value_2": [3.0, 4.0],
    }
).set_index('datetime')

with warnings.catch_warnings(record=True) as w:
    df.plot()
assert 0 == len(w)

[igavronski]

I have patched the local file converter.py using @0x26res suggestion.
In fact, the error message went away, but it calculates the year as 3989...
I replaced line 256 for:

base = dates.EPOCH_OFFSET + np.asarray(dt.asi8 / 1.0E9) / dates.SEC_PER_DAY

[igavronski]

I confirm that modifying the file converter.py (in my Windows 10 installation is located in %USERPROFILE%\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\plotting\_matplotlib),
replacing line 256, by:

base = dates.EPOCH_OFFSET + np.asarray(dt.asi8 / 1.0E9) / dates.SEC_PER_DAY

and inserting the following code in lines 33-34:

import matplotlib.pyplot as plt
plt.rcParams['date.epoch'] = '0000-12-31T00:00:00'

circumvents the problem while developers don't provide a newer, patched code for pd.plot()
Please refer to https://github.com/pandas-dev/pandas/issues/35350#issuecomment-661061337 and https://github.com/pandas-dev/pandas/issues/34850#issuecomment-660965677

[jklymak]

An alternative is matplotlib patches epoch2num to use the new epoch machinery. We didn't use this function internally and didn't test it so derecation seemed reasonable. However if pandas uses it we can fix from our end for back compatibility.

Longer term it might be good to homogenize pandas and matplotlib's date handling. It causes confusion for them to be different, and I suspect that python date handling has matured significantly since either pandas or matplotlib started implementing support.

[TomAugspurger]

I'll try to look into this over the next day or two. Agreed that we should be relying on matplotlib's date handling as much as possible.

[Haaroon]

I confirm that modifying the file converter.py (in my Windows 10 installation is located in %USERPROFILE%\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\pandas\plotting\_matplotlib),
replacing line 256, by:

base = dates.EPOCH_OFFSET + np.asarray(dt.asi8 / 1.0E9) / dates.SEC_PER_DAY

and inserting the following code in lines 33-34:

import matplotlib.pyplot as plt
plt.rcParams['date.epoch'] = '0000-12-31T00:00:00'

circumvents the problem while developers don't provide a newer, patched code for pd.plot()
Please refer to #35350 (comment) and #34850 (comment)

@igavronski Temp workaround fixed the issue for the time being.

[jklymak]

I think I fixed this on the Matplotlib side with https://github.com/matplotlib/matplotlib/pull/17983. I also removed the deprecation since its not fair to ask your users to filter the warning. OTOH, I think that in general, we would prefer to not maintain epoch2num, and num2epoch.

[satishm75]

In my case, I found converter.py at the following location:

C:\Users\USERNAME\AppData\Local\Programs\Python\Python37\Lib\site-packages\pandas\plotting_matplotlib
Added import statement in the import section and plt.rcParams['date.epoch'] = '0000-12-31T00:00:00' in the constant section.

[chrisconlan]

As of July 21st, installing the latest pandas and matplotlib allows plots to work, but the dates are screwed up on the x-axis for datetime-indexed data frames. Installing Matplotlib 3.2.2 as opposed to 3.3.0 fixes the x-axis formatting for now.

[TomAugspurger]

Could people post reproducible examples of plots that look different between mpl 3.2.2 and 3.3.0 with pandas 1.0.5?

[chrisconlan]

import pandas as pd
import matplotlib.pyplot as plt
import datetime
from datetime import timedelta

series = pd.Series(list(range(10000)), index=[datetime.datetime.now() + timedelta(hours=i, minutes=i) for i in range(10000)])
series.plot()
plt.show()

This figure ...

• Python 3.7
• Pandas 1.0.5
• Matplotlib 3.3.0
• Raises lib/python3.7/site-packages/pandas/plotting/_matplotlib/converter.py:256: MatplotlibDeprecationWarning: The epoch2num function was deprecated in Matplotlib 3.3 and will be removed two minor releases later. base = dates.epoch2num(dt.asi8 / 1.0e9)

This figure ...

• Python 3.7
• Pandas 1.0.5
• Matplotlib 3.2.2