Xarray: Reshape function works only for numpy array and raise error with xarray variable.

[ahmedshaaban1]

Code Sample.
raw is xarray.Variable (time: 12784, lat: 73, lon: 144).
I am trying to reshape it.

>>> tmp= np.reshape(raw,(np.shape(raw)[0],np.shape(raw)[1]*np.shape(raw)[2]));
Traceback (most recent call last):

  File "<ipython-input-29-745611cba19c>", line 1, in <module>
    tmp= np.reshape(raw,(np.shape(raw)[0],np.shape(raw)[1]*np.shape(raw)[2]));

  File "/Users/ahmedlasheen/anaconda2/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 257, in reshape
    return _wrapfunc(a, 'reshape', newshape, order=order)

  File "/Users/ahmedlasheen/anaconda2/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 62, in _wrapfunc
    return _wrapit(obj, method, *args, **kwds)

  File "/Users/ahmedlasheen/anaconda2/lib/python2.7/site-packages/numpy/core/fromnumeric.py", line 46, in _wrapit
    result = wrap(result)

  File "/Users/ahmedlasheen/anaconda2/lib/python2.7/site-packages/xarray/core/variable.py", line 1639, in __array_wrap__
    return Variable(self.dims, obj)

  File "/Users/ahmedlasheen/anaconda2/lib/python2.7/site-packages/xarray/core/variable.py", line 259, in __init__
    self._dims = self._parse_dimensions(dims)

  File "/Users/ahmedlasheen/anaconda2/lib/python2.7/site-packages/xarray/core/variable.py", line 420, in _parse_dimensions
    % (dims, self.ndim))

ValueError: dimensions (u'time', u'lat', u'lon') must have the same length as the number of data dimensions, ndim=2

Problem description

I got the previous error, which disappears once I use np.numpy(raw) instead of raw.

Expected Output

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.14.final.0
python-bits: 64
OS: Darwin
OS-release: 18.2.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

xarray: 0.10.9
pandas: 0.23.4
numpy: 1.14.2
scipy: 1.0.0
netCDF4: 1.3.1
h5netcdf: None
h5py: 2.7.1
Nio: None
zarr: None
cftime: None
PseudonetCDF: None
rasterio: None
iris: None
bottleneck: 1.2.1
cyordereddict: None
dask: 0.16.1
distributed: 1.20.2
matplotlib: 2.1.2
cartopy: 0.16.0
seaborn: 0.8.1
setuptools: 38.4.0
pip: 9.0.1
conda: 4.4.10
pytest: 3.3.2
IPython: 5.4.1
sphinx: 1.6.6

[jhamman]

@ahmedshaaban1 - what is the expected output you'd like to get from a reshape call. You may already know that xarray doesn't have a reshape method. Usually, stack/unstack are used to "reshape" DataArrays.

[shoyer]

We should probably remove the xarray.Variable.__array_wrap__ method

[ahmedshaaban1]

Thanks a lot for your answer. I am a novice to Xarray, and I am a heavy user of NumPy. Numpy FFT function works fine with Xarray variables, yet it returns NumPy array. NumPy reshape function, as mentioned above, does not work with Xarray variable. I am not yet able to get the big picture of how to use the power of Xarray and at the same time be able to use the math/statistics function of Numpy/SciPy.

[shoyer]

In general you need to apply NumPy/scipy functions on the .data attribute
of an xarray object, and recreate the labels.
On Thu, Jan 17, 2019 at 10:54 PM Ahmed Shaaban [e-mail]
wrote:

Thanks a lot for your answer. I am a novice to Xarray, and I am a heavy
user of NumPy. Numpy FFT function works fine with Xarray variables, yet it
returns NumPy array. NumPy reshape function, as mentioned above, does not
work with Xarray variable. I am not yet able to understand how to use the
power of Xarray and at the same time be able to use the math/statistics
function of Numpy/SciPy.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/pydata/xarray/issues/2676#issuecomment-455328352, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABKS1tukf4ORIr1y47XKOTaQF10b06aNks5vEOL5gaJpZM4aAzSK
.

[jhamman]

@ahmedshaaban1 - You may also want to checkout http://xarray.pydata.org/en/stable/computation.html#wrapping-custom-computation

@shoyer - can you expand on what you meant about removing the xarray.Variable.__array_wrap__ method? xarray.DataArray also calls this method a few times so we'd have to think about how far to take this.

[shoyer]

__array_wrap__ is an old interface that a variety of NumPy functions use
(in an apparently hard to predict ways).
On Fri, Jan 18, 2019 at 10:07 PM Joe Hamman [e-mail]
wrote:

@ahmedshaaban1 https://github.com/ahmedshaaban1 - You may also want to
checkout
http://xarray.pydata.org/en/stable/computation.html#wrapping-custom-computation

@shoyer https://github.com/shoyer - can you expand on what you meant
about removing the xarray.Variable.__array_wrap__ method? xarray.DataArray
also calls this method a few times so we'd have to think about how far to
take this.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/pydata/xarray/issues/2676#issuecomment-455703570, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABKS1vCyEkGabclgpupG-dyLfz1byEBgks5vEkW5gaJpZM4aAzSK
.

[ahmedshaaban1]

Thanks ...
I am wondering if I used Numpy/Scipy functions on Xarray.data, will such computation is performed under the umbrella of Dask (Lazy evaluation)?

[shoyer]

Generally you'll need to use Dask functions on .data if you want it to be
lazy.
On Sun, Jan 20, 2019 at 9:56 PM Ahmed Shaaban [e-mail]
wrote:

Thanks ...
I am wondering if I used Numpy/Scipy functions on Xarray.data, will such
computation is performed under the umbrella of Dask (Lazy evaluation)?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pydata/xarray/issues/2676#issuecomment-455906272, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABKS1v2mqUzXsYGyLJhSfGH7ZjggyGmbks5vFOYmgaJpZM4aAzSK
.

[ahmedshaaban1]

Any idea if Xarray will incorporate more Numpy/Scipy functions in the future.

[dcherian]

You should also check out xr-scipy which wraps a lot of scipy stuff. And also xrft for Fourier transforms

[ahmedshaaban1]

@dcherian Thanks a lot.