Pandas: BUG: Offset-based rolling window, with closed='left' and max as aggregation function, make python crash

Created on 2 Jul 2018  路  5Comments  路  Source: pandas-dev/pandas

Code Sample, a copy-pastable example if possible

import pandas as pd
import numpy as np
from pandas.tseries.frequencies import to_offset

df = pd.Series(data=np.arange(10), 
               index=pd.date_range('2000', periods=10))

df.rolling(to_offset('3D'), closed='left').max()

Problem description

With this rolling and aggregation function, python just crashes. It does too with .min() or .agg(np.max) as aggregation steps. It does not when closed='right' or with mean as an aggregation function.

Expected Output

2000-01-01    NaN
2000-01-02    0.0
2000-01-03    1.0
2000-01-04    2.0
2000-01-05    3.0
2000-01-06    4.0
2000-01-07    5.0
2000-01-08    6.0
2000-01-09    7.0
2000-01-10    8.0
Freq: D, dtype: float64

or, if closed='left' raises complicated problems (as it is not implemented for fixed windows), disable it too for the offset-based windows.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Windows
OS-release: 7
machine: AMD64
processor: Intel64 Family 6 Model 61 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.1
pytest: 3.2.1
pip: 10.0.1
setuptools: 36.5.0.post20170921
Cython: 0.26.1
numpy: 1.12.1
scipy: 0.19.1
pyarrow: None
xarray: None
IPython: 6.1.0
sphinx: 1.6.3
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.1.0
openpyxl: 2.4.8
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.0
bs4: 4.5.3
html5lib: 0.9999999
sqlalchemy: 1.1.13
pymysql: None
psycopg2: 2.7.3.2 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

Bug Window

Most helpful comment

The bug exists on master branch.

All 5 comments

The bug exists on master branch.

Thanks for the report - that does look strange. Investigation and PRs are always welcome!

Same thing happen with closed='neither', but it works for closed='right'

[Disclaimer : I don't know much about Cython]
The way Python crashes surely mean that it comes from a cython segmentation fault. And the fact that closed='right' works but not left may come from an off-by-one error. As of why the aggregation function matters, I have no clue.
Would love to investigate but as of today I do not have enough experience in C or Cython.

I just hit this as well. @JacquotLeHaricot you're intuition was spot on. Definitely an off-by-one.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

matthiasroder picture matthiasroder  路  3Comments

scls19fr picture scls19fr  路  3Comments

ebran picture ebran  路  3Comments

venuktan picture venuktan  路  3Comments

nathanielatom picture nathanielatom  路  3Comments