Pandas: Empty dataset: typerror ufunc add cannot use operands with types dtype('<M8[ns]') and dtype('O')

[VelizarVESSELINOV]

Code Sample, a copy-pastable example if possible

from pandas import DataFrame, to_datetime, to_timedelta


data = {'Item': [1], 'Date_Time': ['2/8/2015  6:00:30'], 'Relative_Time': 20}
dtf = DataFrame(data=data)

print(dtf)
dtf = dtf[dtf.Item == 2]  # if the filter returns empty dataset expected results will be that the same

dtf.Date_Time = to_datetime(dtf.Date_Time, infer_datetime_format=True)
dtf.Relative_Time = to_timedelta(dtf.Relative_Time, unit='s')
dtf.Date_Time = dtf.Date_Time + dtf.Relative_Time
print(dtf)

Current Output

          Date_Time  Item  Relative_Time
0  2/8/2015  6:00:30     1             20
/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/ops.py:477: PerformanceWarning: Adding/subtracting array of DateOffsets to Series not vectorized
  "Series not vectorized", PerformanceWarning)
Traceback (most recent call last):
  File "bug.py", line 12, in <module>
    dtf.Date_Time = dtf.Date_Time + dtf.Relative_Time
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/ops.py", line 641, in wrapper
    arr = na_op(lvalues, rvalues)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/pandas/core/ops.py", line 481, in <lambda>
    self.na_op = lambda x, y: getattr(x, self.name)(y)
TypeError: ufunc add cannot use operands with types dtype('<M8[ns]') and dtype('O')

Expected Output

          Date_Time  Item  Relative_Time
0  2/8/2015  6:00:30     1             20
Empty DataFrame
Columns: [Date_Time, Item, Relative_Time]
Index: []

output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.5.1.final.0
python-bits: 64
OS: Darwin
OS-release: 15.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.18.1
nose: None
pip: 8.1.2
setuptools: 24.0.3
Cython: None
numpy: 1.11.1
scipy: 0.16.1
statsmodels: None
xarray: None
IPython: 4.0.1
sphinx: None
patsy: None
dateutil: 2.5.3
pytz: 2016.6
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.0
openpyxl: 2.3.5
xlrd: 1.0.0
xlwt: 1.0.0
xlsxwriter: None
lxml: None
bs4: 4.4.1
html5lib: None
httplib2: 0.9.2
apiclient: 1.5.0
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None
None
statsmodels: None
xarray: None
IPython: 4.0.1
sphinx: None
patsy: None
dateutil: 2.5.3
pytz: 2016.6
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.0
openpyxl: 2.3.5
xlrd: 1.0.0
xlwt: 1.0.0
xlsxwriter: None
lxml: None
bs4: 4.4.1
html5lib: None
httplib2: 0.9.2
apiclient: 1.5.0
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None
pandas_datareader: None
None

[jreback]

Here is a simpler replication.

In [63]: df = pd.DataFrame({'A' : Series(dtype='M8[ns]'), 'B' : Series(dtype='m8[ns]')})

In [64]: df
Out[64]: 
Empty DataFrame
Columns: [A, B]
Index: []

In [65]: df.dtypes
Out[65]: 
A     datetime64[ns]
B    timedelta64[ns]
dtype: object

In [66]: df.A+df.B
TypeError: ufunc add cannot use operands with types dtype('<M8[ns]') and dtype('O')

Yeah this empty case is prob address by coercion, maybe @sinhrks has a better idea.

[sinhrks]

The op is performed in _TimeOp class, and it regards empty input as offset even if it has a dtype.

• https://github.com/pydata/pandas/blob/master/pandas/core/ops.py#L596

Then, offset is coerced to object here.

• https://github.com/pydata/pandas/blob/master/pandas/core/ops.py#L532

Fixing _TimeOp._is_offset should solve the problem. PR is appreciated!

[tom-bird]

Happy to take a look at this and submit a PR

[tom-bird]

So, we shouldn't regard something as offset if its empty? This is a three line fix, and certainly resolves the error in the above two examples.