ESMValTool 2.2.0 code freeze

[jvegasbsc]

Now that the conda build is fixed, we can resume the release procedure.

I will suggest to do the code freeze tomorrow at 16:00 to give us a bit more time to do the reviews that both the issues with the core release and the conda build problems prevented us to focus on. Any objections

@ESMValGroup/esmvaltool-coreteam

[bouweandela]

If you postpone the code freeze, I would recommend also postponing the release by the same amount, to make sure we have a full week of testing.

[bouweandela]

Now that the conda build is fixed,

I noticed that there is no mention in the documentation that the conda package will only work for Python 3.7 and up. I think it would be good to add that.

[valeriupredoi]

Now that the conda build is fixed,

I noticed that there is no mention in the documentation that the conda package will only work for Python 3.7 and up. I think it would be good to add that.

good call, I'll add that right now :+1: -> #2030

[jvegasbsc]

If you postpone the code freeze, I would recommend also postponing the release by the same amount, to make sure we have a full week of testing.

That was my intention. I should have made it explicit, though

[jvegasbsc]

Code freeze done! The branch release_2.2.0 is now available for testing.

Please, edit this comment with the recipes that you are going to test to avoid duplication of efforts. It will be nice if you also specify where are you testing them.

List of recipes to test:

• [ ] recipe_albedolandcover.yml
• [x] recipe_anav13jclim.yml (@bettina-gier , mistral)
• [ ] recipe_arctic_ocean.yml
• [x] recipe_autoassess_radiation_rms_Amon_all.yml
• [x] recipe_autoassess_radiation_rms_Amon_obs.yml
• [x] recipe_autoassess_radiation_rms_cfMon_all.yml
• [x] recipe_autoassess_stratosphere.yml -- works only with https://github.com/ESMValGroup/ESMValTool/pull/2047 - now merged so OK
• [x] recipe_capacity_factor.yml (@jvegasbsc, Jasmin)
• [x] recipe_carvalhais14nat.yml (@remi-kazeroni , DKRZ)
• [x] recipe_climwip_test_basic.yml (@ruthlorenz, ETH )
• [x] recipe_climwip_test_performance_sigma.yml (@ruthlorenz, ETH)
• [x] recipe_clouds_bias.yml
• [ ] recipe_clouds_ipcc.yml
• [x] recipe_collins13ipcc.yml (@ruthlorenz, ETH)
• [x] recipe_combined_indices.yml
• [ ] recipe_consecdrydays.yml
• [x] recipe_cox18nature.yml (@schlunma, DKRZ)
• [ ] recipe_cvdp.yml
• [x] recipe_deangelis15nat.yml (@katjaweigel Mistral)
• [ ] recipe_diurnal_temperature_index.yml
• [x] recipe_eady_growth_rate.yml (@sloosvel, Jasmin)
• [x] recipe_ecs.yml (@schlunma, DKRZ)
• [x] recipe_ecs_constraints.yml (@schlunma, DKRZ)
• [x] recipe_ecs_scatter.yml (@schlunma, DKRZ)
• [ ] recipe_ensclus.yml
• [x] recipe_extreme_events.yml (@jvegasbsc, Jasmin)
• [ ] recipe_extreme_index.yml
• [x] recipe_eyring06jgr.yml (@hb326, mistral)
• [x] recipe_eyring13jgr_12.yml (@hb326, mistral)
• [ ] recipe_flato13ipcc.yml
• [x] recipe_gier2020bg.yml (@bettina-gier , mistral)
• [ ] recipe_heatwaves_coldwaves.yml
• [ ] recipe_hyint.yml
• [ ] recipe_hyint_extreme_events.yml
• [ ] recipe_kcs.yml
• [ ] recipe_landcover.yml
• [ ] recipe_lauer13jclim.yml
• [x] recipe_li17natcc.yml (@katjaweigel Mistral)
• [x] recipe_martin18grl.yml (@katjaweigel Mistral)
• [x] recipe_miles_block.yml (@remi-kazeroni , DKRZ)
• [x] recipe_miles_eof.yml (@remi-kazeroni , DKRZ)
• [x] recipe_miles_regimes.yml (@remi-kazeroni , DKRZ)
• [ ] recipe_modes_of_variability.yml
• [ ] recipe_multimodel_products.yml
• [ ] recipe_ocean_Landschuetzer2016.yml
• [ ] recipe_ocean_amoc.yml
• [ ] recipe_ocean_bgc.yml
• [ ] recipe_ocean_example.yml
• [ ] recipe_ocean_ice_extent.yml
• [ ] recipe_ocean_multimap.yml
• [ ] recipe_ocean_quadmap.yml
• [ ] recipe_ocean_scalar_fields.yml
• [ ] recipe_perfmetrics_CMIP5.yml
• [ ] recipe_perfmetrics_CMIP5_4cds.yml
• [x] recipe_perfmetrics_land_CMIP5.yml
• [ ] recipe_quantilebias.yml
• [x] recipe_rainfarm.yml
• [ ] recipe_runoff_et.yml
• [ ] recipe_russell18jgr.yml
• [x] recipe_schlund20esd.yml (@schlunma, DKRZ)
• [ ] recipe_seaice.yml
• [ ] recipe_seaice_drift.yml (@jvegasbsc , Jasmin. Missing key observations, taking too much time to download)
• [x] recipe_seaice_feedback.yml (@jvegasbsc , Jasmin)
• [x] recipe_shapeselect.yml (@remi-kazeroni , DKRZ)
• [x] recipe_smpi.yml (@bettina-gier , mistral)
• [ ] recipe_smpi_4cds.yml
• [ ] recipe_snowalbedo.yml
• [x] recipe_spei.yml
• [x] recipe_tcr.yml (@schlunma, DKRZ)
• [ ] recipe_thermodyn_diagtool.yml
• [x] recipe_toymodel.yml
• [ ] recipe_validation.yml (@remi-kazeroni , DKRZ)
• [ ] recipe_validation_CMIP6.yml
• [ ] recipe_wenzel14jgr.yml
• [ ] recipe_wenzel16jclim.yml
• [ ] recipe_wenzel16nat.yml
• [ ] recipe_williams09climdyn_CREM.yml
• [x] recipe_zmnam.yml

Cmorizers

• [ ] cmorizers/recipe_era5-land.yml
• [ ] cmorizers/recipe_era5.yml

Examples

• [ ] examples/recipe_check_obs.yml
• [x] examples/recipe_concatenate_exps.yml (@schlunma, DKRZ)
• [x] examples/recipe_correlation.yml (@bouweandela asked @esmvalbot at DKRZ https://github.com/ESMValGroup/ESMValTool/pull/2046#issuecomment-782158207)
• [x] examples/recipe_extract_shape.yml (@remi-kazeroni , DKRZ)
• [x] examples/recipe_julia.yml (@bouweandela, laptop, @jvegasbsc jasmin)
• [x] examples/recipe_my_personal_diagnostic.yml (@bouweandela asked @esmvalbot at DKRZ https://github.com/ESMValGroup/ESMValTool/pull/2046#issuecomment-782181075)
• [x] examples/recipe_ncl.yml (@jvegasbsc, Jasmin)
• [ ] examples/recipe_preprocessor_derive_test.yml
• [x] examples/recipe_preprocessor_test.yml (@remi-kazeroni , DKRZ)
• [x] examples/recipe_python.yml (@schlunma, DKRZ)
• [x] examples/recipe_r.yml (@remi-kazeroni , DKRZ)
• [x] examples/recipe_variable_groups.yml (@schlunma, DKRZ)

Hydrology

• [x] hydrology/recipe_hype.yml (@SarahAlidoost)
• [x] hydrology/recipe_lisflood.yml (@SarahAlidoost)
• [x] hydrology/recipe_marrmot.yml (@SarahAlidoost)
• [x] hydrology/recipe_pcrglobwb.yml (@SarahAlidoost)
• [x] hydrology/recipe_wflow.yml (@SarahAlidoost)

Schlund20Jgr

• [x] schlund20jgr/recipe_schlund20jgr_gpp_abs_rcp85.yml (@schlunma, DKRZ)
• [x] schlund20jgr/recipe_schlund20jgr_gpp_change_1pct.yml (@schlunma, DKRZ)
• [x] schlund20jgr/recipe_schlund20jgr_gpp_change_rcp85.yml (@schlunma, DKRZ)

[valeriupredoi]

@ESMValGroup/esmvaltool-developmentteam we have now entered the testing phase before v2.2.0 release, and we have frozen the code (insert song from Frozen here :grin: ) so we'd very much appreciate it if you could run your favourite recipe from the list above, and tell Javi which one you will be running by adding your name against the recipe(s) in the comment. Cheers muchly in advance :beer:

[SarahAlidoost]

The hydrology recipe runs using release_2.2.0 branch were successful. For more info, please see issue #2034.

[remi-kazeroni]

I'm getting an error for the recipe_validation.yml which appears while running the zonal_mean diagnostic. Here is the end of the zonal_mean/log.txt:

2021-02-22 09:32:25,218 [11970] INFO     _validation.py,49      Observations dataset(s) ['CERES-EBAF']
2021-02-22 09:32:25,218 [11970] INFO     validation.py,159      Loading /mnt/lustre01/scratch/b/b309192/runs/tests/esmvaltool_output/recipe_validation_2\
0210222_093105/preproc/validation_with_CERES-EBAF/rsut/CMIP5_MPI-ESM-LR_Amon_historical_r1i1p1_rsut_2001-2005.nc for seasonal extraction
2021-02-22 09:32:25,329 [11970] WARNING  py.warnings,109        /work/bd0854/b309192/soft/miniconda3/envs/test_tool_release/lib/python3.9/site-packages/\
iris/coords.py:1960: UserWarning: Collapsing a non-contiguous coordinate. Metadata may not be fully descriptive for 'time'.
  warnings.warn(msg.format(self.name()))

Traceback (most recent call last):
  File "/mnt/lustre01/pf/b/b309192/code/ESMValTool/esmvaltool/diag_scripts/validation.py", line 335, in <module>
    main(config)
  File "/mnt/lustre01/pf/b/b309192/code/ESMValTool/esmvaltool/diag_scripts/validation.py", line 291, in main
    ctrl_seasons = apply_seasons(ctrl)
  File "/mnt/lustre01/pf/b/b309192/code/ESMValTool/esmvaltool/diag_scripts/validation.py", line 163, in apply_seasons
    season_meaned_cubes = [
  File "/mnt/lustre01/pf/b/b309192/code/ESMValTool/esmvaltool/diag_scripts/validation.py", line 164, in <listcomp>
    season_cube.collapsed('time', iris.analysis.MEAN)
AttributeError: 'NoneType' object has no attribute 'collapsed'

The same issue occurs with meridional_mean. Is this something known? It seems this recipe was succesfully tested for the last release.

[valeriupredoi]

cheers @remi-kazeroni - lemme have a look into this :+1:

[valeriupredoi]

right - I replicated your issue @remi-kazeroni and found the bugger - the implementation of extract_seasonbroke with the custom season implementation, all we have to do is pass meteorological seasons = ["DJF", "MAM", "JJA", "SON"] and that's it, but as it is now the seasons look like seasons = ['DJF', 'MAMJJASON']

[valeriupredoi]

@jvegasbsc that's an issue introduced in https://github.com/ESMValGroup/ESMValCore/pull/247/files#diff-ec2da1a859c8d6359e23421a3de9d444e455ed3008152eaa67bbdd01f4cadfb8 - how did that pass review and how did it not have a test attached to it to reveal its flaw in functionality?

[jvegasbsc]

@remi-kazeroni , this should be fixed with ESMValGroup/ESMValCore#1019. Do we need a 2.2.1 release of the core or we just modify the code? To make it work you just need to add manually the clim_season and season_year coords manually using

iris.coord_categorisation.add_season(cube, 'time', name='clim_season')
iris.coord_categorisation.add_season_year(cube, 'time', name='season_year')

before calling extract_season

[jvegasbsc]

We have done quite a nice bit of testing and only found the issue with the validation_recipe. Once #2054 is merged, which I want to do after the bot finishes or somebody can confirm that it works at DKRZ, I think we can proceed with the release

[jvegasbsc]

v2.2.0 is now published on PyPI and conda. I will proceed to test the conda package, as for some reason the installation is not as smooth as we would like, to make sure that it is functional and then will close the issue. If any of you can also test it, I will be very grateful

[jvegasbsc]

Ok. I have tested a bunch of recipes and they worked. So we can say that v2.2.0 is officially released. Thanks to all for your effort, and specially to V! I owe you quite a few 🍻 !

[valeriupredoi]

awesome, absolute legend @jvegasbsc :medal_military: