Esmvaltool: tests/unit/diag_scripts/mlr/test_preprocess.py fails on select machines

[valeriupredoi]

Summary

Said test fails on select machines e.g. on Jasmin:

(esmvaltool-21) [valeriu@sci2 esmvaltool]$ pytest -n 2 -v tests/unit/diag_scripts/mlr/test_preprocess.py
=================================================== test session starts ===================================================
platform linux -- Python 3.8.6, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /home/users/valeriu/anaconda3R/envs/esmvaltool-21/bin/python3.8
cachedir: .pytest_cache
metadata: {'Python': '3.8.6', 'Platform': 'Linux-3.10.0-1127.19.1.el7.x86_64-x86_64-with-glibc2.10', 'Packages': {'pytest': '6.1.1', 'py': '1.9.0', 'pluggy': '0.13.1'}, 'Plugins': {'flake8': '1.0.6', 'metadata': '1.10.0', 'forked': '1.3.0', 'xdist': '2.1.0', 'env': '0.6.2', 'html': '2.1.1', 'cov': '2.10.1', 'mock': '3.3.1'}}
rootdir: /home/users/valeriu/esmvaltool, configfile: setup.cfg
plugins: flake8-1.0.6, metadata-1.10.0, forked-1.3.0, xdist-2.1.0, env-0.6.2, html-2.1.1, cov-2.10.1, mock-3.3.1
[gw0] linux Python 3.8.6 cwd: /home/users/valeriu/esmvaltool
[gw1] linux Python 3.8.6 cwd: /home/users/valeriu/esmvaltool
[gw1] Python 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 19:08:05)  -- [GCC 7.5.0]
[gw0] Python 3.8.6 | packaged by conda-forge | (default, Oct  7 2020, 19:08:05)  -- [GCC 7.5.0]
gw0 [6] / gw1 [6]
scheduling tests via LoadScheduling

tests/unit/diag_scripts/mlr/test_preprocess.py::FLAKE8 
tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope[x_arr0-y_arr0-3.14] 
[gw0] [ 16%] PASSED tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope[x_arr0-y_arr0-3.14] 
tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope[x_arr2-y_arr2-0.25] 
[gw0] [ 33%] PASSED tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope[x_arr2-y_arr2-0.25] 
tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope_vectorized[x_arr0-y_arr0-output0] 
[gw1] [ 50%] PASSED tests/unit/diag_scripts/mlr/test_preprocess.py::FLAKE8 
tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope[x_arr1-y_arr1-nan] 
[gw1] [ 66%] PASSED tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope[x_arr1-y_arr1-nan] 
[gw0] [ 83%] FAILED tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope_vectorized[x_arr0-y_arr0-output0] 
tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope_vectorized[x_arr1-y_arr1-output1] 
[gw0] [100%] PASSED tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope_vectorized[x_arr1-y_arr1-output1] 

======================================================== FAILURES =========================================================
____________________________________ test_get_slope_vectorized[x_arr0-y_arr0-output0] _____________________________________
[gw0] linux -- Python 3.8.6 /home/users/valeriu/anaconda3R/envs/esmvaltool-21/bin/python3.8

x_arr = array([0, 1, 2, 3, 4])
y_arr = masked_array(
  data=[[[0.0, 2.1, 4.2, 6.300000000000001, 8.4],
         [-0.0, -3.14, -6.28, -9.42, -12.56],
        ...lse],
         [False, False, False, False, False],
         [ True,  True,  True,  True, False]]],
  fill_value=1e+20)
output = array([[ 2.1       , -3.14      ,  0.8       ],
       [-0.46428571, -0.4       ,         nan]])

    @pytest.mark.parametrize('x_arr,y_arr,output', TEST_GET_SLOPE_VECTORIZED)
    def test_get_slope_vectorized(x_arr, y_arr, output):
        """Test vectorized calculation of slope."""
        get_slope = np.vectorize(preprocess._get_slope, excluded=['x_arr'],
                                 signature='(n),(n)->()')
        out = get_slope(x_arr, y_arr)
>       assert ((out == output) | (np.isnan(output) & np.isnan(output))).all()
E       AssertionError: assert False
E        +  where False = <built-in method all of numpy.ndarray object at 0x7f06ffd0fa30>()
E        +    where <built-in method all of numpy.ndarray object at 0x7f06ffd0fa30> = (array([[ 2.1 ...        nan]]) == array([[ 2.1 ...        nan]])
E             +array([[ 2.1       , -3.14      ,  0.8       ],\n
E             +       [-0.46428571, -0.4       ,         nan]])
E             -array([[ 2.1       , -3.14      ,  0.8       ],\n
E             -       [-0.46428571, -0.4       ,         nan]])
E             Full diff:
E               array([[ 2.1       , -3.14      ,  0.8       ],
E                      [-0.46428571, -0.4       ,         nan]],...
E             
E             ...Full output truncated (2 lines hidden), use '-vv' to show | (array([[False, False, False],\n       [False, False,  True]]) & array([[False, False, False],\n       [False, False,  True]]))).all
E        +      where array([[False, False, False],\n       [False, False,  True]]) = <ufunc 'isnan'>(array([[ 2.1       , -3.14      ,  0.8       ],\n       [-0.46428571, -0.4       ,         nan]]))
E        +        where <ufunc 'isnan'> = np.isnan
E        +      and   array([[False, False, False],\n       [False, False,  True]]) = <ufunc 'isnan'>(array([[ 2.1       , -3.14      ,  0.8       ],\n       [-0.46428571, -0.4       ,         nan]]))
E        +        where <ufunc 'isnan'> = np.isnan

tests/unit/diag_scripts/mlr/test_preprocess.py:43: AssertionError
==================================================== warnings summary =====================================================
../anaconda3R/envs/esmvaltool-21/lib/python3.8/site-packages/pyke/knowledge_engine.py:28
../anaconda3R/envs/esmvaltool-21/lib/python3.8/site-packages/pyke/knowledge_engine.py:28
  /home/users/valeriu/anaconda3R/envs/esmvaltool-21/lib/python3.8/site-packages/pyke/knowledge_engine.py:28: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/warnings.html
------------------- generated html file: file:///home/users/valeriu/esmvaltool/test-reports/report.html -------------------

----------- coverage: platform linux, python 3.8.6-final-0 -----------
Coverage HTML written to dir test-reports/coverage_html
Coverage XML written to file test-reports/coverage.xml

================================================= short test summary info =================================================
FAILED tests/unit/diag_scripts/mlr/test_preprocess.py::test_get_slope_vectorized[x_arr0-y_arr0-output0] - AssertionError...
======================================== 1 failed, 5 passed, 2 warnings in 28.96s ============

Note that on my laptop and GA machine it doesn't (they are both Ubuntu, whereas Jasmin is CentOS, see below); also I checked the environments and none of the sfoftware packages differ in version for Jasmin vs my laptop.

Jasmin OS (test fails) vs my OS (test doesnt fail)

• Jasmin:

$ lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.8.2003 (Core)
Release:    7.8.2003
Codename:   Core

• me laptop:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:    18.04
Codename:   bionic

[bouweandela]

Looks like a typo in the test, it should probably be:

assert ((out == output) | (np.isnan(out) & np.isnan(output))).all()

[valeriupredoi]

assert ((out == output) | (np.isnan(out) & np.isnan(output))).all()

nope, still fails - the x and y arrays are different, defo having to do with how the machine computes them, hence the test passing on select systems :+1:

[schlunma]

Can you replace these lines:

https://github.com/ESMValGroup/ESMValTool/blob/6fa4b199f7cc9d30752b7e909e9de092df3a406d/tests/unit/diag_scripts/mlr/test_preprocess.py#L20

https://github.com/ESMValGroup/ESMValTool/blob/6fa4b199f7cc9d30752b7e909e9de092df3a406d/tests/unit/diag_scripts/mlr/test_preprocess.py#L43

with

assert (np.isclose(out, output) | (np.isnan(out) & np.isnan(output))).all()

?

I cannot test myself since the tests are not failing for me.

[valeriupredoi]

@schlunma do you have access to Jasmin? They're only failing there (coz of the OS most prob), or alternatively if you can try running on a CentOS - I will try make the change and test in a jiffy :beer:

[schlunma]

@schlunma do you have access to Jasmin? They're only failing there (coz of the OS most prob), or alternatively if you can try running on a CentOS - I will try make the change and test in a jiffy beer

No, I don't :grimacing: Also no access to another CentOS machine...or is there an easy way to do this? Maybe using docker?

[valeriupredoi]

good news! with that change the test passes :partying_face:

[schlunma]

Awesome!!

[bouweandela]

No, I don't grimacing Also no access to another CentOS machine...or is there an easy way to do this? Maybe using docker?

@schlunma Yes, just run

docker run -it centos:latest

or have a look at dockerhub for the other available versions of CentOS: https://hub.docker.com/_/centos

[valeriupredoi]

@schlunma you gunna put that fix in a PR or you want me to do it? :beer:

[schlunma]

It would be nice if you can do it, you already made the change :smile:

[valeriupredoi]

lazybum! OK shall do :beer: