Cupy: Swapping values in array yields incomplete result

[jakirkham]

When trying to swap 2 values in a CuPy array, it appears that the swap does not proceed to completion. As a result an intermediate state of the swap is stored as the final result.

• Conditions (you can just paste the output of python -c 'import cupy; cupy.show_config()')
  
  
  
  • CuPy version: 6.6.0
  
  
  • OS/Platform: Ubuntu 18.04.2 LTS (bionic)
  
  
  • CUDA version: 10.1.243
  
  
  • cuDNN/NCCL version (if applicable): 7.6.4, 2.4.8.1
  
  

CuPy Version          : 6.6.0
CUDA Root             : /usr/local/cuda
CUDA Build Version    : 10010
CUDA Driver Version   : 10010
CUDA Runtime Version  : 10010
cuDNN Build Version   : 7604
cuDNN Version         : 7604
NCCL Build Version    : 2406
NCCL Runtime Version  : 2408

• Code to reproduce

In [1]: import cupy as cp                                                       

In [2]: a = cp.arange(10)                                                       

In [3]: a                                                                       
Out[3]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [4]: a[0], a[1] = a[1], a[0]                                                 

In [5]: a                                                                       
Out[5]: array([1, 1, 2, 3, 4, 5, 6, 7, 8, 9])

• Error messages, stack traces, or logs: NA

cc @pentschev

[emcastillo]

The problem seems to be when assigning a view of the array to the same array.
In the example below, b holds references and not actual copies of the elements array.
When the __setitem__ function of the array is called to do the assignment for the first element, it modifies the memory of the second element in the b tuple. So the second time is called, the value has been already replaced 😢. Support such assignment will require device synchronization or modifying cupy code in non-trivial ways so we have decided to leave this as-is.

b = (a[1], a[0])
a[0], a[1] = b

Fails while

b = (cp.array(1), cp.array(0))
a[0], a[1] = b

works

In numpy the same behavior holds if you do:

a[0:1], a[1:2] = a[1:2], a[0:1]

[toslunar]

In order to achieve the swap, a[[0, 1]] = a[[1, 0]] looks good to me.

[benfred]

Fwiw - this problem also means that we get incorrect results from passing a cupy array to np.random.shuffle:

arr = cupy.arange(10)
# arr = array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

np.random.shuffle(arr)
# arr = array([0, 1, 1, 3, 1, 5, 6, 3, 1, 2]) 

numpy .random.shuffle internally is just swapping elements: https://github.com/numpy/numpy/blob/9f12028b9453c17b72b26355fd503e512af96a5d/numpy/random/_generator.pyx#L4455-L4458 which triggers this issue

[jakirkham]

Yeah Ben I believe that was the underlying use case that motivated this issue

Also NumPy issue ( https://github.com/numpy/numpy/issues/15123 ) is tracking dispatching on array type in numpy.random functions