Xarray: TypeError for NetCDF float16 output

Created on 1 Feb 2018  路  4Comments  路  Source: pydata/xarray

ds = xr.Dataset({"test": np.arange(0, 5, dtype='float16')})
ds.to_netcdf('test.nc')

This fails because the float16 type doesn't exist for NetCDF files, throwing an TypeError: illegal primitive data type.

It might be nice if the xarray netCDF engine promoted this to float32 instead.

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Linux
OS-release: 3.11.0-26-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_GB.UTF-8
LOCALE: en_GB.UTF-8

xarray: 0.10.0
pandas: 0.22.0
numpy: 1.14.0
scipy: 1.0.0
netCDF4: 1.3.1
h5netcdf: 0.5.0
Nio: None
bottleneck: 1.2.1
cyordereddict: None
dask: 0.16.1
matplotlib: 2.1.2
cartopy: 0.15.1
seaborn: 0.8.1
setuptools: 38.4.0
pip: 9.0.1
conda: None
pytest: None
IPython: 6.2.1
sphinx: None

backends

Most helpful comment

It might be nice if the xarray netCDF engine promoted this to float32 instead.

I would generally lean against this type of behavior. Isn't it nice to know that float16 isn't supported rather than being surprised later when you discover your data was automatically upcast?

All 4 comments

It might be nice if the xarray netCDF engine promoted this to float32 instead.

I would generally lean against this type of behavior. Isn't it nice to know that float16 isn't supported rather than being surprised later when you discover your data was automatically upcast?

I suppose we could do while issuing a warning. Using encoding={'test': {'dtype': np.float32}} in to_netcdf() would be the way to silence the warning.

Do we have a verdict here? I'm personally fine raising an error (perhaps a better one than the current) unless an encoding/dtype is provided.

It's OK - I think @shoyer is right - it shouldn't silently promote the type, and the error message is clear. Thanks

Was this page helpful?
0 / 5 - 0 ratings