Esmvaltool: Using dask 2 results in a stack overflow

Created on 1 Jul 2019  路  34Comments  路  Source: ESMValGroup/ESMValTool

Running examples/recipe_python.yml in a new installation results in a crash. This can be solved by using version 1 of dask (version 2 was released just a few days ago). I'm not sure whether this is a problem with dask or the way iris uses dask.

2019-07-01 14:29:40,353 [7436] INFO     diagnostic.py,89    Processing variable air_temperature
2019-07-01 14:29:40,353 [7436] INFO     diagnostic.py,91    Processing dataset CanESM2
2019-07-01 14:29:40,353 [7436] DEBUG    diagnostic.py,41    Loading /home/bandela/esmvaltool_output/recipe_python_20190701_141848/preproc/diagnostic1/ta/CMIP5_CanESM2_Amon_historical_r1i1p1_ta_2000-2002.nc
2019-07-01 14:29:40,405 [7436] DEBUG    diagnostic.py,44    Running example computation
Fatal Python error: Cannot recover from stack overflow.

Thread 0x00007f2b6effd700 (most recent call first):
  File "/home/bandela/conda/envs/test/lib/python3.7/multiprocessing/pool.py", line 470 in _handle_results
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 865 in run
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 917 in _bootstrap_inner
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 885 in _bootstrap

Thread 0x00007f2b6f7fe700 (most recent call first):
  File "/home/bandela/conda/envs/test/lib/python3.7/multiprocessing/pool.py", line 422 in _handle_tasks
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 865 in run
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 917 in _bootstrap_inner
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 885 in _bootstrap

Thread 0x00007f2b6ffff700 (most recent call first):
  File "/home/bandela/conda/envs/test/lib/python3.7/multiprocessing/pool.py", line 413 in _handle_workers
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 865 in run
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 917 in _bootstrap_inner
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 885 in _bootstrap

Thread 0x00007f2b7cf38700 (most recent call first):
  File "/home/bandela/conda/envs/test/lib/python3.7/multiprocessing/pool.py", line 110 in worker
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 865 in run
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 917 in _bootstrap_inner
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 885 in _bootstrap

Thread 0x00007f2b9d6f7700 (most recent call first):
  File "/home/bandela/conda/envs/test/lib/python3.7/multiprocessing/pool.py", line 110 in worker
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 865 in run
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 917 in _bootstrap_inner
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 885 in _bootstrap

Thread 0x00007f2b9cef6700 (most recent call first):
  File "/home/bandela/conda/envs/test/lib/python3.7/multiprocessing/pool.py", line 110 in worker
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 865 in run
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 917 in _bootstrap_inner
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 885 in _bootstrap

Thread 0x00007f2b9c6f5700 (most recent call first):
  File "/home/bandela/conda/envs/test/lib/python3.7/multiprocessing/pool.py", line 110 in worker
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 865 in run
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 917 in _bootstrap_inner
  File "/home/bandela/conda/envs/test/lib/python3.7/threading.py", line 885 in _bootstrap

Current thread 0x00007f2ba0cc3740 (most recent call first):
  File "/home/bandela/conda/envs/test/lib/python3.7/inspect.py", line 2469 in __init__
  File "/home/bandela/conda/envs/test/lib/python3.7/inspect.py", line 2154 in _signature_from_function
  File "/home/bandela/conda/envs/test/lib/python3.7/inspect.py", line 2284 in _signature_from_callable
  File "/home/bandela/conda/envs/test/lib/python3.7/inspect.py", line 2378 in _signature_from_callable
  File "/home/bandela/conda/envs/test/lib/python3.7/inspect.py", line 2833 in from_callable
  File "/home/bandela/conda/envs/test/lib/python3.7/inspect.py", line 3083 in signature
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 1325 in has_keyword
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 530 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/utils.py", line 854 in __call__
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/utils.py", line 122 in compute_meta
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/blockwise.py", line 233 in blockwise
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 566 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 1952 in map_blocks
  File "/home/bandela/conda/envs/test/lib/python3.7/site-packages/dask/array/core.py", line 2113 in copy
  ...
iris
Source
picture bouweandela

Most helpful comment

OK good newskies! dask=2.1.0 does not produce the stackoverflow error anymore, recipe_python.yml runs through and the slicing is done error-free now:

cube = iris.load_cube("/group_workspaces/jasmin4/ncas_cms/valeriu/strato_data/ta_Amon_UKESM1-0-LL-piCont_piControl_r1i1p1f2_gn_2004-2014.nc")
print(cube[(0, slice(None, None, None), slice(None, None, None), slice(None, None, None))])

@bouweandela we should edit the environment.yml file for dask>=2.1.0

picture valeriupredoi on 11 Jul 2019
🎉1 👍1

All 34 comments

this has been reported to me as well by a user from China (see attached stack report).

I am trying to reproduce the issue but currently the anaconda stack is unreachable (Bad gateway 502). I suspect there is an issue with the newest iris rather than dask; has iris moved up in version?
liangliang.txt

valeriupredoi picture valeriupredoi on 2 Jul 2019

I had the same problem today as well (on DKRZ).

schlunma picture schlunma on 2 Jul 2019

note that I am unable to reproduce the error just by upgrading dask:

$ conda list dask
# packages in environment at /home/valeriu/anaconda3/envs/esmv:
#
# Name                    Version                   Build  Channel
dask                      2.0.0                      py_0    conda-forge
dask-core                 2.0.0                      py_0    conda-forge

and iris has not yet moved up in version,

Creating a brand new environment does not allow replication either. OS is Ubuntu, am trying now on Jasmin's Red Hat

valeriupredoi picture valeriupredoi on 2 Jul 2019

OK it works fine for me on Jasmin with dask=2.0.0 @bouweandela @schlunma can you try again, guys?

valeriupredoi picture valeriupredoi on 2 Jul 2019

Nahh I spoke too soon - the diagnostic itself is dying with Fatal Python error: Cannot recover from stack overflow.

valeriupredoi picture valeriupredoi on 2 Jul 2019

this is the offending bit: cube.collapsed('time', iris.analysis.MEAN)

valeriupredoi picture valeriupredoi on 2 Jul 2019

OK I found the problem - it is collapsing on time that is causing it, specifically: in cube.py in func: collapsed

        for dim in dims_to_collapse:
            indices[dim] = 0
        collapsed_cube = self[tuple(indices)]

it has nothing to do with the cube per se but rather the cube slicing ie that slicing is done like

cube[(0, slice(None, None, None), slice(None, None, None), slice(None, None, None))]

even if you lad a (any) cube and try slicing it like that you get the stack overflow error.

That is from dask.slicing that has changed quite a bit since dask=1.1.2 so I wouldnt be able to point to what's causing it. More importantly it seems to be an issue specific to dask and not iris

valeriupredoi picture valeriupredoi on 2 Jul 2019

cross-assigning Bill @bjlittle since this is a dask via iris in esmvaltool problem :beer:

valeriupredoi picture valeriupredoi on 3 Jul 2019
👍1

@valeriupredoi Thanks. I'll discuss this with the dask guys. For the interim, we've pinned dask<2 oniris master, otherwise our Travis CI fails.

bjlittle picture bjlittle on 9 Jul 2019
👍1

@valeriupredoi dask has just release 2.1.0... I'm re-running our iris Travis CI tests and unpinning dask to use the latest version (we just recently pinned iris to dask<2 to avoid the stack overflow issue in Travis CI).

It seems to be working for us - there are some new issues to resolve when using dask 2.1.0 under iris, but at first glance this just seems to involve us updating a couple of our tests. I certainly can't recreate the stack overflow on Travic CI with the latest dask.

@valeriupredoi @bouweandela @schlunma Can you try dask 2.1.0 and see if this resolves the issue on your side?

bjlittle picture bjlittle on 11 Jul 2019
🎉2

@bjlittle ace, man, cheers for the heads up! will test today! :beer:

valeriupredoi picture valeriupredoi on 11 Jul 2019
👍1

OK good newskies! dask=2.1.0 does not produce the stackoverflow error anymore, recipe_python.yml runs through and the slicing is done error-free now:

cube = iris.load_cube("/group_workspaces/jasmin4/ncas_cms/valeriu/strato_data/ta_Amon_UKESM1-0-LL-piCont_piControl_r1i1p1f2_gn_2004-2014.nc")
print(cube[(0, slice(None, None, None), slice(None, None, None), slice(None, None, None))])

@bouweandela we should edit the environment.yml file for dask>=2.1.0

valeriupredoi picture valeriupredoi on 11 Jul 2019
🎉1 👍1

cheers @bjlittle :beer:

valeriupredoi picture valeriupredoi on 11 Jul 2019
👍1

@valeriupredoi Happy days :wink:

I guess you guys can close this issue down. BTW all our iris Travis CI tests are now passing using dask 2.1.0.

@valeriupredoi Are you okay if I now close down your associated iris issue https://github.com/SciTools/iris/issues/3346?

bjlittle picture bjlittle on 11 Jul 2019

@valeriupredoi Happy days wink

I guess you guys can close this issue down. BTW all our iris Travis CI tests are now passing using dask 2.1.0.

sweet! we'll close this when we've edited and pushed the version requirement in the environment file

@valeriupredoi Are you okay if I now close down your associated iris issue SciTools/iris#3346?

Absoluteness! no reason to keep it at your end if yous happy :beer:

valeriupredoi picture valeriupredoi on 11 Jul 2019
❤1

@bjlittle a word of warning - installing iris from conda via conda install -c conda-forge iris will not pick up the latest dask, in fact it will pick up a very ye olde version - is this what you guys want?

valeriu@valeriu-PORTEGE-Z30-C:~$ conda list iris
# packages in environment at /home/valeriu/anaconda3:
#
# Name                    Version                   Build  Channel
iris                      2.2.1                    py36_0    conda-forge
valeriu@valeriu-PORTEGE-Z30-C:~$ conda list dask
# packages in environment at /home/valeriu/anaconda3:
#
# Name                    Version                   Build  Channel
dask                      0.16.1                   py36_0  
dask-core                 0.16.1                   py36_0
valeriupredoi picture valeriupredoi on 11 Jul 2019
😕1

note: this installation is not done part of an environment creation, it is the simple-Joe install iris for the first time on a machine

valeriupredoi picture valeriupredoi on 11 Jul 2019

I have tested the environment creation and installation of esmvaltool and it does pick up dask=2.1.0 (as the latest version) w/o pinning it in the environment file; tool installs correctly and runs fine, so am gonna close this - @bjlittle have a look at my pointer above for installing iris as a stand alone package and its dask, but as far as we're concerned this is a-OK since all stuffs is inside virtenvs for us :beer:

valeriupredoi picture valeriupredoi on 11 Jul 2019

@valeriupredoi Thanks.... slightly confused though. For me I get...

> conda create -n iris-test -c conda-forge iris
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /tmp/persistent/itwl/miniconda3/envs/iris-test

  added / updated specs:
    - iris


The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
  antlr-python-runt~ conda-forge/linux-64::antlr-python-runtime-4.7.2-py37_1000
  asn1crypto         conda-forge/linux-64::asn1crypto-0.24.0-py37_1003
  bokeh              conda-forge/linux-64::bokeh-1.2.0-py37_0
  bzip2              conda-forge/linux-64::bzip2-1.0.6-h14c3975_1002
  ca-certificates    conda-forge/linux-64::ca-certificates-2019.6.16-hecc5488_0
  cartopy            conda-forge/linux-64::cartopy-0.17.0-py37h0aa2c8f_1004
  certifi            conda-forge/linux-64::certifi-2019.6.16-py37_0
  cf-units           conda-forge/linux-64::cf-units-2.1.1-py37h3010b51_1000
  cffi               conda-forge/linux-64::cffi-1.12.3-py37h8022711_0
  cftime             conda-forge/linux-64::cftime-1.0.3.4-py37hd352d35_1001
  chardet            conda-forge/linux-64::chardet-3.0.4-py37_1003
  click              conda-forge/noarch::click-7.0-py_0
  cloudpickle        conda-forge/noarch::cloudpickle-1.2.1-py_0
  cryptography       conda-forge/linux-64::cryptography-2.7-py37h72c5cf5_0
  curl               conda-forge/linux-64::curl-7.64.1-hf8cf82a_0
  cycler             conda-forge/noarch::cycler-0.10.0-py_1
  cytoolz            conda-forge/linux-64::cytoolz-0.9.0.1-py37h14c3975_1001
  dask               conda-forge/noarch::dask-2.1.0-py_0
  dask-core          conda-forge/noarch::dask-core-2.1.0-py_0
  dbus               conda-forge/linux-64::dbus-1.13.6-he372182_0
  distributed        conda-forge/noarch::distributed-2.1.0-py_0
  expat              conda-forge/linux-64::expat-2.2.5-he1b5a44_1003
  fontconfig         conda-forge/linux-64::fontconfig-2.13.1-he4413a7_1000
  freetype           conda-forge/linux-64::freetype-2.10.0-he983fc9_0
  geos               conda-forge/linux-64::geos-3.7.1-hf484d3e_1000
  gettext            conda-forge/linux-64::gettext-0.19.8.1-hc5be6a0_1002
  glib               conda-forge/linux-64::glib-2.58.3-h6f030ca_1001
  gst-plugins-base   conda-forge/linux-64::gst-plugins-base-1.14.5-h0935bb2_0
  gstreamer          conda-forge/linux-64::gstreamer-1.14.5-h36ae1b5_0
  hdf4               conda-forge/linux-64::hdf4-4.2.13-h9a582f1_1002
  hdf5               conda-forge/linux-64::hdf5-1.10.5-nompi_h3c11f04_1100
  heapdict           conda-forge/linux-64::heapdict-1.0.0-py37_1000
  icu                conda-forge/linux-64::icu-58.2-hf484d3e_1000
  idna               conda-forge/linux-64::idna-2.8-py37_1000
  iris               conda-forge/linux-64::iris-2.2.1-py37_0
  jinja2             conda-forge/noarch::jinja2-2.10.1-py_0
  jpeg               conda-forge/linux-64::jpeg-9c-h14c3975_1001
  kiwisolver         conda-forge/linux-64::kiwisolver-1.1.0-py37hc9558a2_0
  krb5               conda-forge/linux-64::krb5-1.16.3-h05b26f9_1001
  libblas            conda-forge/linux-64::libblas-3.8.0-10_openblas
  libcblas           conda-forge/linux-64::libcblas-3.8.0-10_openblas
  libcurl            conda-forge/linux-64::libcurl-7.64.1-hda55be3_0
  libedit            conda-forge/linux-64::libedit-3.1.20170329-hf8c457e_1001
  libffi             conda-forge/linux-64::libffi-3.2.1-he1b5a44_1006
  libgcc-ng          pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0
  libgfortran-ng     pkgs/main/linux-64::libgfortran-ng-7.3.0-hdf63c60_0
  libiconv           conda-forge/linux-64::libiconv-1.15-h516909a_1005
  liblapack          conda-forge/linux-64::liblapack-3.8.0-10_openblas
  libnetcdf          conda-forge/linux-64::libnetcdf-4.6.2-h056eaf5_1002
  libopenblas        conda-forge/linux-64::libopenblas-0.3.6-h6e990d7_4
  libpng             conda-forge/linux-64::libpng-1.6.37-hed695b0_0
  libssh2            conda-forge/linux-64::libssh2-1.8.2-h22169c7_2
  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
  libtiff            conda-forge/linux-64::libtiff-4.0.10-h57b8799_1003
  libuuid            conda-forge/linux-64::libuuid-2.32.1-h14c3975_1000
  libxcb             conda-forge/linux-64::libxcb-1.13-h14c3975_1002
  libxml2            conda-forge/linux-64::libxml2-2.9.9-h13577e0_1
  locket             conda-forge/noarch::locket-0.2.0-py_2
  lz4-c              conda-forge/linux-64::lz4-c-1.8.3-he1b5a44_1001
  markupsafe         conda-forge/linux-64::markupsafe-1.1.1-py37h14c3975_0
  matplotlib         pkgs/main/linux-64::matplotlib-2.2.3-py37hb69df0a_0
  msgpack-python     conda-forge/linux-64::msgpack-python-0.6.1-py37h6bb024c_0
  ncurses            conda-forge/linux-64::ncurses-6.1-hf484d3e_1002
  netcdf4            conda-forge/linux-64::netcdf4-1.5.1.2-py37h73a1b54_1
  numpy              conda-forge/linux-64::numpy-1.16.4-py37h95a1406_0
  olefile            conda-forge/noarch::olefile-0.46-py_0
  openblas           conda-forge/linux-64::openblas-0.3.6-h6e990d7_4
  openssl            conda-forge/linux-64::openssl-1.1.1c-h516909a_0
  owslib             conda-forge/noarch::owslib-0.18.0-py_0
  packaging          conda-forge/noarch::packaging-19.0-py_0
  pandas             conda-forge/linux-64::pandas-0.24.2-py37hb3f55d8_0
  partd              conda-forge/noarch::partd-1.0.0-py_0
  pcre               conda-forge/linux-64::pcre-8.41-hf484d3e_1003
  pillow             conda-forge/linux-64::pillow-6.1.0-py37he7afcd5_0
  proj4              conda-forge/linux-64::proj4-5.2.0-he1b5a44_1003
  psutil             conda-forge/linux-64::psutil-5.6.3-py37h516909a_0
  pthread-stubs      conda-forge/linux-64::pthread-stubs-0.4-h14c3975_1001
  pycparser          conda-forge/linux-64::pycparser-2.19-py37_1
  pyepsg             conda-forge/noarch::pyepsg-0.4.0-py_0
  pykdtree           conda-forge/linux-64::pykdtree-1.3.1-py37h3010b51_1002
  pyke               conda-forge/linux-64::pyke-1.1.1-py37_1000
  pyopenssl          conda-forge/linux-64::pyopenssl-19.0.0-py37_0
  pyparsing          conda-forge/noarch::pyparsing-2.4.0-py_0
  pyproj             conda-forge/linux-64::pyproj-1.9.6-py37h516909a_1002
  pyqt               conda-forge/linux-64::pyqt-5.9.2-py37hcca6a23_0
  pyshp              conda-forge/noarch::pyshp-2.1.0-py_0
  pysocks            conda-forge/linux-64::pysocks-1.7.0-py37_0
  python             conda-forge/linux-64::python-3.7.3-h33d41f4_1
  python-dateutil    conda-forge/noarch::python-dateutil-2.8.0-py_0
  pytz               conda-forge/noarch::pytz-2019.1-py_0
  pyyaml             conda-forge/linux-64::pyyaml-5.1.1-py37h516909a_0
  qt                 conda-forge/linux-64::qt-5.9.7-h52cfd70_2
  readline           conda-forge/linux-64::readline-8.0-hf8c457e_0
  requests           conda-forge/linux-64::requests-2.22.0-py37_0
  scipy              conda-forge/linux-64::scipy-1.3.0-py37h921218d_0
  setuptools         conda-forge/linux-64::setuptools-41.0.1-py37_0
  shapely            conda-forge/linux-64::shapely-1.6.4-py37h06cd6f9_1005
  sip                conda-forge/linux-64::sip-4.19.8-py37hf484d3e_1000
  six                conda-forge/linux-64::six-1.12.0-py37_1000
  sortedcontainers   conda-forge/noarch::sortedcontainers-2.1.0-py_0
  sqlite             conda-forge/linux-64::sqlite-3.29.0-hcee41ef_0
  tblib              conda-forge/noarch::tblib-1.4.0-py_0
  tk                 conda-forge/linux-64::tk-8.6.9-hed695b0_1002
  toolz              conda-forge/noarch::toolz-0.9.0-py_1
  tornado            conda-forge/linux-64::tornado-6.0.3-py37h516909a_0
  udunits2           conda-forge/linux-64::udunits2-2.2.27.6-h4e0c4b3_1001
  urllib3            conda-forge/linux-64::urllib3-1.24.3-py37_0
  xorg-libxau        conda-forge/linux-64::xorg-libxau-1.0.9-h14c3975_0
  xorg-libxdmcp      conda-forge/linux-64::xorg-libxdmcp-1.1.3-h516909a_0
  xz                 conda-forge/linux-64::xz-5.2.4-h14c3975_1001
  yaml               conda-forge/linux-64::yaml-0.1.7-h14c3975_1001
  zict               conda-forge/noarch::zict-1.0.0-py_0
  zlib               conda-forge/linux-64::zlib-1.2.11-h14c3975_1004
  zstd               conda-forge/linux-64::zstd-1.4.0-h3b9ef0a_0

So this pulls in iris 2.2.1 and dask 2.1.0.

Note quite sure what's going on at your end. Perhaps there is an ESMValTool centric dependency that is pulling dask down to an older version?

BTW I've found that it's best to blow away old conda environments, and start afresh, as sometime you can get some package dependency oddness.

I'm assuming that you can get the latest iris and dask in a new conda environment, right?

bjlittle picture bjlittle on 11 Jul 2019

note: this installation is not done part of an environment creation, it is the simple-Joe install iris for the first time on a machine

Bill, installing iris within an environment pulls the latest iris and dask; when installing iris as a stand alone package (no environment involved), just when you want to get iris only:

[joepesci@italian-computer]$ conda list iris
#
# Name                    Version                   Build  Channel
[joepesci@italian-computer]$ conda install -c conda-forge iris
valeriupredoi picture valeriupredoi on 11 Jul 2019

OK I repeated the process from the point of a brand new install of anaconda:

[valeriu@jasmin-sci2 ~]$ bash Anaconda3-2019.03-Linux-x86_64.sh
[valeriu@jasmin-sci2 ~]$ export PATH=/home/users/valeriu/anaconda3/bin:$PATH
[valeriu@jasmin-sci2 ~]$ conda update -n base conda
[valeriu@jasmin-sci2 ~]$ conda install -c conda-forge iris
[valeriu@jasmin-sci2 ~]$ conda list iris
# packages in environment at /home/users/valeriu/anaconda3:
#
# Name                    Version                   Build  Channel
iris                      2.2.0                 py37_1003    conda-forge
[valeriu@jasmin-sci2 ~]$ conda list dask
# packages in environment at /home/users/valeriu/anaconda3:
#
# Name                    Version                   Build  Channel
dask                      1.1.4                    py37_1  
dask-core                 1.1.4                    py37_1  
[valeriu@jasmin-sci2 ~]$ conda --version
conda 4.7.5

so iris 2.2.1 and dask 2.1.0 are not defaults for basic installations outside environments...

valeriupredoi picture valeriupredoi on 11 Jul 2019

@valeriupredoi Sorry, but I can't recreate this behaviour.

I've downloaded the latest Miniconda3, then installed iris from conda-forge and I get iris 2.2.1 and dask 2.1.0.

Note quite sure what's going on at your end - can't quite explain it. Conda will always get the latest version of a package, if a pin is not provided. As you can see, the latest version of iris on conda-forge is 2.2.1, see here.

bjlittle picture bjlittle on 11 Jul 2019
👍1

You don't have anything weird set-up in you ~/.condarc ?

bjlittle picture bjlittle on 11 Jul 2019

What happens if you try to do:

> conda install -c conda-forge iris=2.2.1
bjlittle picture bjlittle on 11 Jul 2019

@valeriupredoi Sorry, but I can't recreate this behaviour.

I've downloaded the latest Miniconda3, then installed iris from conda-forge and I get iris 2.2.1 and dask 2.1.0.

Note quite sure what's going on at your end - can't quite explain it. Conda will always get the latest version of a package, if a pin is not provided. As you can see, the latest version of iris on conda-forge is 2.2.1, see here.

yes, totally awares of conda's workflow, hence a bit confused as to why it's doing this

I don't have a ~/.condarc

Let's just forget about it, unless there are other users reporting such issues, I'll call it some weird stuff happening at my end, which doesn't affect me a bit so yeah, forget about it :grin:

valeriupredoi picture valeriupredoi on 11 Jul 2019
👍1

btw actually try anaconda rather than miniconda, I found out what's the weird stuff at my end (latest Anaconda this is):

[valeriu@jasmin-sci2 esmvaltool]$ conda install -c conda-forge iris=2.2.1
WARNING conda.base.context:use_only_tar_bz2(632): Conda is constrained to only using the old .tar.bz2 file format because you have conda-build installed, and it is <3.18.3.  Update or remove conda-build to get smaller downloads and faster extractions.
Collecting package metadata (repodata.json): done
Solving environment: failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

  - anaconda==2019.03=py37_0 -> importlib_metadata==0.8=py37_0
  - anaconda==2019.03=py37_0 -> matplotlib==3.0.3=py37h5429711_0
  - anaconda==2019.03=py37_0 -> pango==1.42.4=h049681c_0
  - cartopy -> matplotlib
  - iris=2.2.1 -> cartopy[version='>=0.14'] -> matplotlib
  - matplotlib
  - pango
  - pkgs/main/linux-64::importlib_metadata==0.8=py37_0
  - pkgs/main/linux-64::path.py==11.5.0=py37_0 -> importlib_metadata[version='>=0.5']
valeriupredoi picture valeriupredoi on 11 Jul 2019

it works well in an environment tho, the base is fussy by the looks of it

valeriupredoi picture valeriupredoi on 11 Jul 2019

Nice detective work :male_detective: ... and good to know!

Cheers fella :+1:

bjlittle picture bjlittle on 11 Jul 2019
😄1

ah I know what it is! it's conda's inability to downgrade matplotlib to 2.2.4 (latest version working with iris) and it's keeping it to 3.0.3 then throwing a fit - and this is why you are not seeing this since miniconda does not come with matplotlib, it gets installed via iris rather than via anaconda

You guys may want to add a note about this on the site somewhere

valeriupredoi picture valeriupredoi on 11 Jul 2019
👍1

iris is pinned to matplotlib >=2,<3 as we're still supporting Python2 and Python3, as matplotlib3+ is Python3 only.

We're scheduling iris 3.0.0, which will be Python3 only on 5 Nov 2019, see https://github.com/scitools/iris/milestones

bjlittle picture bjlittle on 11 Jul 2019
👍1

all sweet and dandy, but make sure you tell your users to use miniconda instead of anaconda so they don't run in what I've just run into (if they don't use virtual envs) (if this is not already in the iris documentation) :beer:

valeriupredoi picture valeriupredoi on 11 Jul 2019

We're scheduling iris 3.0.0, which will be Python3 only on 5 Nov 2019, see https://github.com/scitools/iris/milestones

Wohoo! I remember I asked you at the workshop in Oberpf... if 3.0 is gonna be breaking all sorts of deps for ESMValTool and you said nay...just making sure I get a written account of it as well :rofl:

valeriupredoi picture valeriupredoi on 11 Jul 2019

@valeriupredoi Well, I'll go as far as saying that I'd be surprised if the move to iris 3.0.0 will break any dependencies. If anything it will allow us to open up some dependencies i.e. move to matplotlib 3.x, but I suspect that we'd do that after the release of 3.0.0, say in iris 3.1.0

bjlittle picture bjlittle on 12 Jul 2019
❤1

cheers Bill-san :beer:

valeriupredoi picture valeriupredoi on 15 Jul 2019
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

IreMav picture IreMav  路  4Comments
francesco-cmcc picture francesco-cmcc  路  4Comments
jhardenberg picture jhardenberg  路  5Comments
axel-lauer picture axel-lauer  路  5Comments
bouweandela picture bouweandela  路  4Comments