Incubator-mxnet: [numpy] np.random.multinomial is different from _np

[hzfan]

Description

np.random.multinomial produces result significantly different from _np.random.multinomial

Environment info

python 3.7.3

Minimum reproducible example

from mxnet import np, npx
import numpy as _np
import random

npx.set_np()
p = [1 / 6.] * 6
random.seed(0)
x = _np.random.multinomial(20, p)
for i in range(1000):
  x = x + _np.random.multinomial(20, p)
print(x)

random.seed(0)
x = np.random.multinomial(20, p)
for i in range(1000):
  x = x + np.random.multinomial(20, p)
print(x)

The result is
[3311 3297 3293 3416 3332 3371] [ 1337 1287 1352 1338 1364 13342]

Thank @stu1130 @reminisce for kind help.

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Bug

[stu1130]

Thanks @hzfan I will look into it

[stu1130]

Have narrowed down the root cause is that the instability of SampleUniform I used for loc value

the value below shoule be bounded in [0.0, 1.0].
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.46602e+13
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.54167
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.46602e+13
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.54167
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.46602e+13
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.54167
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.46602e+13
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.54167
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.46602e+13
[12:05:29] src/operator/numpy/random/./np_multinomial_op.h:112: 1.54167

result:
[ 0  0  0  0  0 10]

[stu1130]

created a fix on #15397

[stu1130]

@mxnet-label-bot add [bug]

[stu1130]

@haojin2 the PR was merged, could you close the issue? Thanks

[haojin2]

@hzfan Please verify this on your end and close the issue once you think your issue is solved, thanks

[hzfan]

Thank you for the quick fix. Issue closed.

[TomasBahnik]

I am experiencing somehow similar issue on Windows with mxnet 1.6.0b20190926 (example from d2l book)

from mxnet import np, npx
fair_probs = [1.0 / 6] * 6
np.random.multinomial(10, fair_probs, size=1)
Out[5]: array([[ 0,  0,  0,  0,  0, 10]], dtype=int64)
np.random.multinomial(100, fair_probs, size=1)
Out[6]: array([[  0,   0,   0,   0,   0, 100]], dtype=int64)
np.random.multinomial(10000, fair_probs, size=1)
Out[7]: array([[    0,     0,     0,     0,     0, 10000]], dtype=int64)
np.random.multinomial(10000, fair_probs, size=3)
Out[8]: 
array([[    0,     0,     0,     0,     0, 10000],
       [ 1610,     0,     0,     0,     0,  8390],
       [ 1684,  1710,     0,     0,     0,  6606]], dtype=int64)

[haojin2]

@stu1130 @xidulu

[xidulu]

Cannot reproduce the error on ubuntu.

[stu1130]

cannot reproduce the error on mac with 1.6.0b20190926
I will try it on windows

[vexilligera]

I am experiencing somehow similar issue on Windows with mxnet 1.6.0b20190926 (example from d2l book)

from mxnet import np, npx
fair_probs = [1.0 / 6] * 6
np.random.multinomial(10, fair_probs, size=1)
Out[5]: array([[ 0,  0,  0,  0,  0, 10]], dtype=int64)
np.random.multinomial(100, fair_probs, size=1)
Out[6]: array([[  0,   0,   0,   0,   0, 100]], dtype=int64)
np.random.multinomial(10000, fair_probs, size=1)
Out[7]: array([[    0,     0,     0,     0,     0, 10000]], dtype=int64)
np.random.multinomial(10000, fair_probs, size=3)
Out[8]: 
array([[    0,     0,     0,     0,     0, 10000],
       [ 1610,     0,     0,     0,     0,  8390],
       [ 1684,  1710,     0,     0,     0,  6606]], dtype=int64)

Hi. I cannot reproduce the error with MXNet 1.6 built from source on Windows Server 2016.

>>> from mxnet import np, npx
>>> fair_probs = [1.0 / 6] * 6
>>> np.random.multinomial(10, fair_probs, size=1)
array([[1, 3, 1, 2, 3, 0]], dtype=int64)
>>> np.random.multinomial(100, fair_probs, size=1)
array([[23, 23, 13, 22,  8, 11]], dtype=int64)
>>> np.random.multinomial(10000, fair_probs, size=1)
array([[1687, 1682, 1677, 1618, 1683, 1653]], dtype=int64)
>>> np.random.multinomial(10000, fair_probs, size=3)
array([[1611, 1641, 1685, 1683, 1671, 1709],
       [1682, 1710, 1649, 1647, 1702, 1610],
       [1705, 1661, 1681, 1671, 1688, 1594]], dtype=int64)

[TomasBahnik]

OK. So this only issue on Windows 10 to be more precise.

[vexilligera]

OK. So this only issue on Windows 10 to be more precise.

You can try building from source.

[stu1130]

@TomasBahnik I was able to reproduce the issue with mxnet 1.6.0b20190926 on windows. The pip wheel doesn't have COMMIT_HASH but I guess the codebase is not up-to-date. i.e. it didn't include #16204. I also tried mxnet-mkl 1.6.0b20191004 but no luck the error still exists. I would recommend to build from source before our pipline roll out new wheel.

[TomasBahnik]

Issue is still present in Windows mxnet 1.6.0b20191125 updated today by pip install -U --pre mxnet

[haojin2]

@TomasBahnik Sorry, just discovered that the fix is somehow not reflected in the current master branch, will verify the cause and deliver a fix.

[xidulu]

@TomasBahnik
@stu1130
I was able to reproduce the error on windows with mxnet 1.6.0b20191125 (this is actually the latest version for Windows, even though 1.60b2020xxxx has already been available for other platforms), installed by pip install -U --pre mxnet.

I also tested np.random.uniform and everything seems to good, which suggests that the random number generator is working normally. Also, I tried np.ndarray as the input format for pvals and the output was still sth like [0,0,0,10]. The ndarray case should not be relevant with either #16204 or #15397 IMO, so the cause could be something other than "the fix commits are not included".

[stu1130]

@xidulu Thanks for deep dive. I would probably get back to this a few weeks later

[jarlostensen]

This is still an issue, with 1.18.4. am I right? At least I am seeing it with mxnet 1.16.