Esmvaltool: Test esmvalcore=2.3.0 with current batch of ESMValTool recipes

Created on 15 Jun 2021  路  25Comments  路  Source: ESMValGroup/ESMValTool

Hey good peeps @ESMValGroup/esmvaltool-developmentteam we have released esmvalcore=2.3.0 yesterday but we didn't do a sanity check and run all the recipes from ESMValTool, there were some key changes pushed forward with the release and we have decided to perform this check anyway, just to be sure all's still up and running. So could we please ask you to grab one or more of your favourite recipes and run it with ESMValCore installed from either main or PyPi or conda, your choice - only be sure to have pulled the latest main and/or the latest package off PyPi/conda. A fresh installation of ESMValTool will bring forth the newly released ESMValCore if you remove the pin - esmvalcore>=2.2.0,<2.3 on 2.3 in environment.yml and setup.py. Cheers ever so much! Oh and feel free to use the bot if @nielsdrost could install the latest esmvalcore there? There will be weissbiers as rewards for you at the next (actual, 3D, face-to-face) meeting at DLR :grin: :beer:

  • [ ] recipe_albedolandcover.yml
  • [x] recipe_anav13jclim.yml (@schlunma)
  • [ ] recipe_arctic_ocean.yml
  • [ ] recipe_autoassess_radiation_rms_Amon_all.yml
  • [ ] recipe_autoassess_radiation_rms_Amon_obs.yml
  • [ ] recipe_autoassess_radiation_rms_cfMon_all.yml
  • [x] recipe_autoassess_stratosphere.yml -- but only with matplotlib pin from https://github.com/ESMValGroup/ESMValTool/pull/2200
  • [ ] recipe_bock20jgr.yml
  • [ ] recipe_capacity_factor.yml
  • [ ] recipe_carvalhais14nat.yml
  • [x] recipe_climwip_test_basic.yml (@lukasbrunner )
  • [x] recipe_climwip_test_performance_sigma.yml (@lukasbrunner )
  • [ ] recipe_clouds_bias.yml
  • [ ] recipe_clouds_ipcc.yml
  • [ ] recipe_collins13ipcc.yml
  • [ ] recipe_combined_indices.yml
  • [ ] recipe_consecdrydays.yml
  • [x] recipe_cox18nature.yml (@schlunma) [runs with #2203]
  • [ ] recipe_cvdp.yml
  • [ ] recipe_deangelis15nat.yml (@katjaweigel )
  • [ ] recipe_diurnal_temperature_index.yml
  • [x] recipe_eady_growth_rate.yml
  • [x] recipe_ecs_constraints.yml (@schlunma) [runs with #2203]
  • [ ] recipe_ecs_scatter.yml (@schlunma)
  • [x] recipe_ecs.yml (@schlunma) [runs with #2203]
  • [ ] recipe_ensclus.yml
  • [ ] recipe_extreme_events.yml
  • [ ] recipe_extreme_index.yml
  • [ ] recipe_eyring06jgr.yml
  • [ ] recipe_eyring13jgr_12.yml
  • [ ] recipe_flato13ipcc.yml
  • [x] recipe_gier2020bg.yml (@bettina-gier )
  • [ ] recipe_heatwaves_coldwaves.yml
  • [ ] recipe_hyint_extreme_events.yml
  • [ ] recipe_hyint.yml
  • [ ] recipe_impact.yml
  • [ ] recipe_kcs.yml (@remi-kazeroni)
  • [ ] recipe_landcover.yml
  • [ ] recipe_lauer13jclim.yml
  • [x] recipe_li17natcc.yml (@katjaweigel )
  • [x] recipe_martin18grl.yml (@katjaweigel )
  • [ ] recipe_meehl20sciadv.yml
  • [x] recipe_miles_block.yml (@remi-kazeroni)
  • [ ] recipe_miles_eof.yml
  • [ ] recipe_miles_regimes.yml
  • [ ] recipe_modes_of_variability.yml
  • [ ] recipe_multimodel_products.yml
  • [ ] recipe_ocean_amoc.yml
  • [x] recipe_ocean_bgc.yml (@zklaus)
  • [ ] recipe_ocean_example.yml
  • [ ] recipe_ocean_ice_extent.yml
  • [ ] recipe_ocean_Landschuetzer2016.yml
  • [ ] recipe_ocean_multimap.yml
  • [ ] recipe_ocean_quadmap.yml
  • [ ] recipe_ocean_scalar_fields.yml
  • [ ] recipe_perfmetrics_CMIP5_4cds.yml
  • [x] recipe_perfmetrics_CMIP5.yml (@bettina-gier )
  • [ ] recipe_perfmetrics_land_CMIP5.yml
  • [x] recipe_quantilebias.yml (@remi-kazeroni)
  • [ ] recipe_rainfarm.yml
  • [ ] recipe_runoff_et.yml
  • [ ] recipe_russell18jgr.yml
  • [x] recipe_schlund20esd.yml (@schlunma) [runs with #2203]
  • [ ] recipe_seaice_drift.yml
  • [x] recipe_seaice_feedback.yml (@jvegasbsc )
  • [ ] recipe_seaice.yml
  • [ ] recipe_shapeselect.yml
  • [ ] recipe_smpi_4cds.yml
  • [x] recipe_smpi.yml (@bettina-gier )
  • [ ] recipe_snowalbedo.yml
  • [ ] recipe_spei.yml
  • [x] recipe_tcr.yml (@schlunma) [runs with #2203]
  • [x] recipe_thermodyn_diagtool.yml (@katjaweigel )
  • [ ] recipe_toymodel.yml
  • [ ] recipe_validation_CMIP6.yml
  • [ ] recipe_validation.yml
  • [ ] recipe_wenzel14jgr.yml
  • [ ] recipe_wenzel16jclim.yml
  • [ ] recipe_wenzel16nat.yml
  • [ ] recipe_williams09climdyn_CREM.yml
  • [ ] recipe_zmnam.yml
  • [ ] recipe_era5-land.yml
  • [ ] recipe_era5.yml
  • [x] recipe_check_obs.yml (@remi-kazeroni)
  • [x] recipe_concatenate_exps.yml (@remi-kazeroni)
  • [x] recipe_correlation.yml (@remi-kazeroni)
  • [x] recipe_extract_shape.yml (@remi-kazeroni)
  • [ ] recipe_julia.yml
  • [x] recipe_my_personal_diagnostic.yml (@schlunma)
  • [x] recipe_ncl.yml (@remi-kazeroni)
  • [x] recipe_preprocessor_derive_test.yml (@schlunma)
  • [ ] recipe_preprocessor_test.yml
  • [x] recipe_python.yml (@remi-kazeroni)
  • [x] recipe_r.yml (@remi-kazeroni)
  • [ ] recipe_variable_groups.yml
  • [ ] recipe_hydro_forcing.yml
  • [ ] recipe_hype.yml
  • [ ] recipe_lisflood.yml
  • [ ] recipe_marrmot.yml
  • [ ] recipe_pcrglobwb.yml
  • [ ] recipe_wflow.yml
  • [x] recipe_schlund20jgr_gpp_abs_rcp85.yml (@schlunma)
  • [x] recipe_schlund20jgr_gpp_change_1pct.yml (@schlunma) [runs with ESMValGroup/ESMValCore#1188]
  • [x] recipe_schlund20jgr_gpp_change_rcp85.yml (@schlunma)
picture valeriupredoi

Most helpful comment

@remi-kazeroni,

  • The first error you mention seems to be connected to fx files. @sloosvel, could you please have a look?
  • The second error comes about because in that recipe almost all datasets are deactivated. When you activate at least one other dataset, the recipe should work; I had the same behavior. If we want to allow the application of multi-model statistics to a single dataset (a bit non-sensical, but why not?) I suggest opening a specific issue about that.
picture zklaus on 17 Jun 2021
👍4

All 25 comments

The usual procedure is to update the ESMValTool repository so it uses the latest version of ESMValCore immediately after the ESMValcore release. That way there are two weeks in which diagnostic developers can try out if their recipes work with the latest version of the ESMValCore before we release ESMValTool.

Oh and feel free to use the bot if @nielsdrost could install the latest esmvalcore there?

The bot uses the branch in the ESMValTool repository that you're requesting a test for, so if that branch uses the latest version of ESMValCore, the bot will also use it.

Should we move this issue to the ESMValTool repository, as it is about testing recipes in ESMValTool?

bouweandela picture bouweandela on 16 Jun 2021
🎉1 👍1

Should we move this issue to the ESMValTool repository, as it is about testing recipes in ESMValTool?

yeah, I was thinking of moving it, good point, man!

valeriupredoi picture valeriupredoi on 16 Jun 2021
🎉1

For recipe_schlund20jgr_gpp_change_1pct.yml, I got a

2021-06-17 11:04:44,917 UTC [12752] ERROR   Failed to run fix_metadata([<iris 'Cube' of gross_primary_productivity_of_carbon / (kg m-2 s-1) (time: 1680; latitude: 64; longitude: 128)>], {'project': 'CMIP5', 'short_name': 'gpp', 'mip': 'Lmon', 'exp': 'esmFixClim1', 'ense
mble': 'r1i1p1', 'ref': True, 'preprocessor': 'preproc_total_mean_flux_var', 'variable_group': 'ref', 'diagnostic': 'diag_gpp_fraction_mean', 'dataset': 'MIROC-ESM', 'start_year': 11, 'end_year': 20, 'recipe_dataset_index': 4, 'institute': ['MIROC'], 'alias': 'MIROC-ESM
', 'original_short_name': 'gpp', 'standard_name': 'gross_primary_productivity_of_carbon', 'long_name': 'Carbon Mass Flux out of Atmosphere due to Gross Primary Production on Land', 'units': 'kg m-2 s-1', 'modeling_realm': ['land'], 'frequency': 'mon', 'filename': '/scra
tch/b/b309141/work_v2/recipe_schlund20jgr_gpp_change_1pct_20210617_110434/preproc/diag_gpp_fraction_mean/ref/CMIP5_MIROC-ESM_Lmon_esmFixClim1_r1i1p1_gpp_11-20.nc', 'check_level': <CheckLevels.DEFAULT: 3>})
2021-06-17 11:04:44,962 UTC [12663] INFO    Progress: 8 tasks running, 18 tasks waiting for ancestors, 2/28 done
2021-06-17 11:04:46,176 UTC [12663] INFO    Maximum memory used (estimate): 1.5 GB
2021-06-17 11:04:46,177 UTC [12663] INFO    Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2021-06-17 11:04:46,178 UTC [12663] ERROR   Program terminated abnormally, see stack trace below for more information:
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "cftime/_cftime.pyx", line 505, in cftime._cftime.num2date
OverflowError: date value out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/lustre02/work/bd0854/b309141/miniconda3/envs/esm2/lib/python3.9/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/_task.py", line 754, in _run_task
    output_files = task.run()
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/_task.py", line 252, in run
    self.output_files = self._run(input_files)
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/preprocessor/__init__.py", line 482, in _run
    product.apply(step, self.debug)
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/preprocessor/__init__.py", line 351, in apply
    self.cubes = preprocess(self.cubes, step, **self.settings[step])
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/preprocessor/__init__.py", line 295, in preprocess
    result.append(_run_preproc_function(function, items, settings))
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/preprocessor/__init__.py", line 281, in _run_preproc_function
    return function(items, **kwargs)
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/cmor/fix.py", line 124, in fix_metadata
    cube = checker(cube).check_metadata()
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/cmor/check.py", line 178, in check_metadata
    self._check_time_coord()
  File "/mnt/lustre01/pf/b/b309141/ESMValCore_2/esmvalcore/cmor/check.py", line 764, in _check_time_coord
    first = coord.cell(i).point
  File "/mnt/lustre02/work/bd0854/b309141/miniconda3/envs/esm2/lib/python3.9/site-packages/iris/coords.py", line 1925, in cell
    bound = self.units.num2date(bound)
  File "/mnt/lustre02/work/bd0854/b309141/miniconda3/envs/esm2/lib/python3.9/site-packages/cf_units/__init__.py", line 2036, in num2date
    return _num2date_to_nearest_second(
  File "/mnt/lustre02/work/bd0854/b309141/miniconda3/envs/esm2/lib/python3.9/site-packages/cf_units/__init__.py", line 609, in _num2date_to_nearest_second
    dates = cftime.num2date(
  File "cftime/_cftime.pyx", line 507, in cftime._cftime.num2date
ValueError: OverflowError in datetime, possibly because year < datetime.MINYEAR

This is the affected dataset: /pf/b/b309141/work/CMIP5_DKRZ/MIROC/MIROC-ESM/esmFixClim1/mon/land/Lmon/r1i1p1/v1/gpp/gpp_Lmon_MIROC-ESM_esmFixClim1_r1i1p1_000101-014012.nc

Here is the log file:
OUTPUT_SCHLUNDETAL_GPP_1PCT.txt

schlunma picture schlunma on 17 Jun 2021
🎉1

Thanks for looking into this, @schlunma! For the path you posted I only get Permission denied, so I tried to find it in the regular CMIP5 archive on mistral, but there, no esmFixClim1 experiment exists, only esmFixClim2 and that doesn't contain the same data. Are we sure that this is official, still valid CMIP5 data? ESGF also doesn't list esmFixClim1 for this model.

zklaus picture zklaus on 17 Jun 2021
🎉1

I can't find the data anymore on the official archive - since I got it from there it must have been retracted.

Nevertheless, I found the issue (the first value of the bound of the time coordinate is smaller than datetime.MINYEAR, which is set to 1) and was able to solve this with an easy fix. I would really like to keep the recipe as is and open a PR with the fix. Doesn't need to be included in 2.3., but would be nice if it was in the main branch.

schlunma picture schlunma on 17 Jun 2021

No objection in principle. Just wasn't able to determine what exactly was going on. Do you want to make a separate issue describing the problem?

zklaus picture zklaus on 17 Jun 2021

Yes, will do.

Found another issue, this time in recipe_schlund20esd.yml: Due to the newly added automatic addition of fx files, the tool tries to load areacello now for AWI-CM, which leads to the issue described by @remi-kazeroni at the very bottom in ESMValGroup/ESMValCore#751.

This can be solved by adapting the recipe (with fx_files: {areacella}), but this is something we really should fix in 2.4, since it affects all datasets with unstructured grid (e.g., also ICON data).

schlunma picture schlunma on 17 Jun 2021

I got an error with the examples/recipe_preprocessor_test.yml using a fresh new installation, see below. For comparison, I also ran the recipe with the DKRZ central module (v2.2.0 for the core and the tool) and that worked fine.

2021-06-17 12:47:22,964 UTC [1213] ERROR   esmvalcore.preprocessor:283 Failed to run multi_model_statistics({<esmvalcore.preprocessor.PreprocessorFile object at 0x2ae7134799a0>, <esmvalcore.preprocessor.Preprocess\
orFile object at 0x2ae71346c730>, <esmvalcore.preprocessor.PreprocessorFile object at 0x2ae7134419a0>, <esmvalcore.preprocessor.PreprocessorFile object at 0x2ae713410f70>}, {'span': 'overlap', 'statistics': ['mean\
', 'median'], 'output_products': {'mean': <esmvalcore.preprocessor.PreprocessorFile object at 0x2ae7134a3ca0>, 'median': <esmvalcore.preprocessor.PreprocessorFile object at 0x2ae713431ac0>}})
2021-06-17 12:47:23,715 UTC [1213] INFO    esmvalcore._task:127 Maximum memory used (estimate): 3.8 GB
2021-06-17 12:47:23,715 UTC [1213] INFO    esmvalcore._task:129 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2021-06-17 12:47:23,717 UTC [1213] ERROR   esmvalcore._main:440 Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 433, in run
    fire.Fire(ESMValTool())
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 410, in run
    process_recipe(recipe_file=recipe, config_user=cfg)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 104, in process_recipe
    recipe.run()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_recipe.py", line 1434, in run
    self.tasks.run(max_parallel_tasks=self._cfg['max_parallel_tasks'])
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 674, in run
    self._run_sequential()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 685, in _run_sequential
    task.run()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 252, in run
    self.output_files = self._run(input_files)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 475, in _run
    self.products = _apply_multimodel(self.products, step,
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 417, in _apply_multimodel
    result = preprocess(products - exclude, step, **settings)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 295, in preprocess
    result.append(_run_preproc_function(function, items, settings))
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 281, in _run_preproc_function
    return function(items, **kwargs)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 429, in multi_model_statistics
    return _multiproduct_statistics(
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 340, in _multiproduct_statistics
    statistics_cubes = _multicube_statistics(cubes=cubes,
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 320, in _multicube_statistics
    result_cube = _compute_eager(aligned_cubes,
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 275, in _compute_eager
    combined_slice = _combine(single_model_slices)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 262, in _combine
    merged_cube = cubes.merge_cube()
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/iris/cube.py", line 404, in merge_cube
    proto_cube.register(cube, error_on_mismatch=True)
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/iris/_merge.py", line 1358, in register
    match = cube_signature.match(other, error_on_mismatch)
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/iris/_merge.py", line 465, in match
    raise iris.exceptions.MergeError(msgs)
iris.exceptions.MergeError: failed to merge into a single cube.
  cube.ancillary_variables differ
2021-06-17 12:47:23,813 UTC [1213] INFO    esmvalcore._main:444
If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions
If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues
To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.

main_log_debug.txt

remi-kazeroni picture remi-kazeroni on 17 Jun 2021

I also encountered a problem with the multi_model_statistics preprocessor for the recipe_ocean_bgc.yml. This is not related to the version number of the WOA dataset (see #1812) but rather to the core issue pointed out by @tomaslovato in this comment. For comparison, I could ran this recipe successfully using the DKRZ central module. @zklaus could you please confirm that you ran the recipe successfully with the latest installation? Here is the error I get:

2021-06-17 12:50:23,099 UTC [1212] ERROR   esmvalcore.preprocessor:283 Failed to run multi_model_statistics({<esmvalcore.preprocessor.PreprocessorFile object at 0x2b6929e936d0>}, {'span': 'overlap', 'statistics': \
['mean'], 'output_products': {'mean': <esmvalcore.preprocessor.PreprocessorFile object at 0x2b6929e93070>}})
2021-06-17 12:50:23,813 UTC [1212] INFO    esmvalcore._task:127 Maximum memory used (estimate): 2.1 GB
2021-06-17 12:50:23,814 UTC [1212] INFO    esmvalcore._task:129 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur.
2021-06-17 12:50:23,815 UTC [1212] ERROR   esmvalcore._main:440 Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 433, in run
    fire.Fire(ESMValTool())
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 410, in run
    process_recipe(recipe_file=recipe, config_user=cfg)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 104, in process_recipe
    recipe.run()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_recipe.py", line 1434, in run
    self.tasks.run(max_parallel_tasks=self._cfg['max_parallel_tasks'])
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 674, in run
    self._run_sequential()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 685, in _run_sequential
    task.run()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 248, in run
    input_files.extend(task.run())
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 252, in run
    self.output_files = self._run(input_files)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 475, in _run
    self.products = _apply_multimodel(self.products, step,
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 417, in _apply_multimodel
    result = preprocess(products - exclude, step, **settings)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 295, in preprocess
    result.append(_run_preproc_function(function, items, settings))
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 281, in _run_preproc_function
    return function(items, **kwargs)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 429, in multi_model_statistics
    return _multiproduct_statistics(
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 340, in _multiproduct_statistics
    statistics_cubes = _multicube_statistics(cubes=cubes,
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 309, in _multicube_statistics
    raise ValueError('Cannot perform multicube statistics '
ValueError: Cannot perform multicube statistics for a single cube.
2021-06-17 12:50:23,958 UTC [1212] INFO    esmvalcore._main:444
If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions
If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues
To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.

main_log_debug.txt

EDIT from @valeriupredoi (meself) - ping @Peter9192 :beer:

remi-kazeroni picture remi-kazeroni on 17 Jun 2021

autoassess stratosphere is succombing to a plotting error related to nc-time-axis:

2021-06-17 10:29:19,817 [23078] WARNING  esmvaltool.diag_scripts.autoassess.stratosphere.age_of_air,114 Run length < 12 years: Can't assess age of air
Traceback (most recent call last):
  File "/home/users/valeriu/esmvaltool/esmvaltool/diag_scripts/autoassess/autoassess_area_base.py", line 428, in <module>
    run_area(config)
  File "/home/users/valeriu/esmvaltool/esmvaltool/diag_scripts/autoassess/autoassess_area_base.py", line 420, in run_area
    multi_function(run_obj)
  File "/home/users/valeriu/esmvaltool/esmvaltool/diag_scripts/autoassess/stratosphere/strat_metrics_1.py", line 681, in multi_qbo_plot
    fig.savefig('qbo_30hpa.png')
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/figure.py", line 3005, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/backend_bases.py", line 2255, in print_figure
    result = print_method(
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/backend_bases.py", line 1669, in wrapper
    return func(*args, **kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/backends/backend_agg.py", line 508, in print_png
    return func(*args, **kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/backends/backend_agg.py", line 508, in print_png
    FigureCanvasAgg.draw(self)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/backends/backend_agg.py", line 406, in draw
    self.figure.draw(self.renderer)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/artist.py", line 74, in draw_wrapper
    result = draw(artist, renderer, *args, **kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/artist.py", line 51, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/figure.py", line 2780, in draw
    mimage._draw_list_compositing_images(
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/image.py", line 132, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/artist.py", line 51, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/_api/deprecation.py", line 431, in wrapper
    return func(*inner_args, **inner_kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/axes/_base.py", line 2921, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/image.py", line 132, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/artist.py", line 51, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/_api/deprecation.py", line 431, in wrapper
    return func(*inner_args, **inner_kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/axes/_base.py", line 2921, in draw
    mimage._draw_list_compositing_images(renderer, self, artists)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/image.py", line 132, in _draw_list_compositing_images
    a.draw(renderer)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/artist.py", line 51, in draw_wrapper
    return draw(artist, renderer, *args, **kwargs)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/axis.py", line 1136, in draw
    ticks_to_draw = self._update_ticks()
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/axis.py", line 1023, in _update_ticks
    major_locs = self.get_majorticklocs()
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/matplotlib/axis.py", line 1255, in get_majorticklocs
    return self.major.locator()
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/nc_time_axis/__init__.py", line 151, in __call__
    return self.tick_values(vmin, vmax)
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/nc_time_axis/__init__.py", line 176, in tick_values
    ticks = [
  File "/home/users/valeriu/miniconda3-June2021/envs/release230/lib/python3.9/site-packages/nc_time_axis/__init__.py", line 177, in <listcomp>
    cftime.datetime(
  File "cftime/_cftime.pyx", line 873, in cftime._cftime.datetime.__init__
TypeError: Expected str, got numpy.str_

fairly sure this is due to the new Matplotlib 3.4.2 not liking the older nc-time-axis but nc-time-axis=1.3.1 can not be installed in current configuration, see https://github.com/SciTools/nc-time-axis/issues/71

valeriupredoi picture valeriupredoi on 17 Jun 2021

@remi-kazeroni,

  • The first error you mention seems to be connected to fx files. @sloosvel, could you please have a look?
  • The second error comes about because in that recipe almost all datasets are deactivated. When you activate at least one other dataset, the recipe should work; I had the same behavior. If we want to allow the application of multi-model statistics to a single dataset (a bit non-sensical, but why not?) I suggest opening a specific issue about that.
zklaus picture zklaus on 17 Jun 2021
👍4

@zklaus It might be the case to revise a little the recipe recipe_ocean_bgc.yml by adding a second dataset to pass through the single dataset multimodel lock? Maybe a CMIP6 dataset?

tomaslovato picture tomaslovato on 17 Jun 2021
👍2

The first error you mention seems to be connected to fx files. @sloosvel, could you please have a look?

From the log, it looks like some datasets (bcc-csm1-1, GFDL-ESM2G) lose the ancillary variables after the extract_levels step, whereas others (MPI-ESM-LR and ERA-Interim) do not. And I guess that becomes a problem at the multimodel step.

sloosvel picture sloosvel on 18 Jun 2021

The second error comes about because in that recipe almost all datasets are deactivated. When you activate at least one other dataset, the recipe should work; I had the same behavior. If we want to allow the application of multi-model statistics to a single dataset (a bit non-sensical, but why not?) I suggest opening a specific issue about that.

@zklaus It might be the case to revise a little the recipe recipe_ocean_bgc.yml by adding a second dataset to pass through the single dataset multimodel lock? Maybe a CMIP6 dataset?

Sure, the error can be circumvented by adding another dataset but it would be good to include that in the recipe as @tomaslovato suggests. We may not need to have multi-model statistics working on a single dataset but the recipe becomes unusable in its current form for the new release.

It would also be good that we work on "minimal configurations" for testing the recipes for the releases. At the moment, many of the recipes in the list can't be tested directly (at least on Mistral) because of data availability...

remi-kazeroni picture remi-kazeroni on 18 Jun 2021

@remi-kazeroni, you are completely right. The point of this issue here is to determine if a bug slipped us by in making the core release that necessitates a bugfix release for the core. It is of course always possible that a core release entails changes to the recipes; those are completely fine and should be dealt with in this period between the core release and the tool release.

zklaus picture zklaus on 18 Jun 2021
👍1

Sorry for posting these errors from failing recipes without investigating in detail... This time it is about recipe_kcs.yml. Note that the same recipe runs with the version 2.2.0 of the core.

2021-06-18 06:46:32,482 UTC [30582] ERROR   esmvalcore._main:440 Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 433, in run
    fire.Fire(ESMValTool())
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
    component_trace = _Fire(component, args, parsed_flag_args, context, name)
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire
    component, remaining_args = _CallAndUpdateTrace(
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
    component = fn(*varargs, **kwargs)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 410, in run
    process_recipe(recipe_file=recipe, config_user=cfg)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_main.py", line 104, in process_recipe
    recipe.run()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_recipe.py", line 1434, in run
    self.tasks.run(max_parallel_tasks=self._cfg['max_parallel_tasks'])
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 674, in run
    self._run_sequential()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 685, in _run_sequential
    task.run()
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 248, in run
    input_files.extend(task.run())
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/_task.py", line 252, in run
    self.output_files = self._run(input_files)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 475, in _run
    self.products = _apply_multimodel(self.products, step,
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 417, in _apply_multimodel
    result = preprocess(products - exclude, step, **settings)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 295, in preprocess
    result.append(_run_preproc_function(function, items, settings))
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/__init__.py", line 281, in _run_preproc_function
    return function(items, **kwargs)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 429, in multi_model_statistics
    return _multiproduct_statistics(
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 340, in _multiproduct_statistics
    statistics_cubes = _multicube_statistics(cubes=cubes,
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 320, in _multicube_statistics
    result_cube = _compute_eager(aligned_cubes,
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 275, in _compute_eager
    combined_slice = _combine(single_model_slices)
  File "/mnt/lustre01/pf/b/b309192/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 262, in _combine
    merged_cube = cubes.merge_cube()
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/iris/cube.py", line 404, in merge_cube
    proto_cube.register(cube, error_on_mismatch=True)
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/iris/_merge.py", line 1358, in register
    match = cube_signature.match(other, error_on_mismatch)
  File "/work/bd0854/b309192/soft/miniconda3/envs/release_core/lib/python3.9/site-packages/iris/_merge.py", line 465, in match
    raise iris.exceptions.MergeError(msgs)
iris.exceptions.MergeError: failed to merge into a single cube.
  cube data dtype differs: float32 != float64

main_log_debug.txt

remi-kazeroni picture remi-kazeroni on 18 Jun 2021

There is also a problem in recipe_deangelis15nat.yml with the derived variable lvp. Although the error messages says it could be missing data I don't think that this is the reason, but I'll try to understand what is the issue.
error_ recipe_deangelis15nat_lvp.txt

Units shouldn't be the issue, the equation should take care of this. The error says:
ValueError: Coordinate 'latitude' has different points for the LHS cube 'precipitation_flux' and RHS cube 'water_evaporation_flux'.
What are LHS and RHS cubes?

recipe_li17natcc.yml ran without problems.

katjaweigel picture katjaweigel on 18 Jun 2021

There is also a problem in recipe_deangelis15nat.yml with the derived variable lvp. Although the error messages says it could be missing data I don't think that this is the reason, but I'll try to understand what is the issue.
error_ recipe_deangelis15nat_lvp.txt

Units shouldn't be the issue, the equation should take care of this. The error says:
ValueError: Coordinate 'latitude' has different points for the LHS cube 'precipitation_flux' and RHS cube 'water_evaporation_flux'.
What are LHS and RHS cubes?

recipe_li17natcc.yml ran without problems.

Some model seems to miss a fix to have the same number of digits for the latitude for each variable.
It is MIROC5 and it is probably related to this pull request: https://github.com/ESMValGroup/ESMValCore/pull/1110
There a round_coordinates(cubes) is now also applied to pr. For recipe_deangelis15nat it would be necessary to apply it also to evspsbl and hfls. There could be more variables necessary for other combinations somewhere, but I don't know how to test this?

katjaweigel picture katjaweigel on 18 Jun 2021

recipe_martin18grl.yml runs but I found an issue (probably related to longitude) in some of the plots, (e.g. SPI_mapObservations_Dur_of_Events_Mean.png). But this is not related to the new core release, I checked that it also happend with the old core last year. I'll look at it after the issues related to the release.

katjaweigel picture katjaweigel on 18 Jun 2021

For recipe_deangelis15nat.yml: should I remove MIROC5 or is it possible to do the rounding from ESMValGroup/ESMValCore#1110 also for evspsbl and hfls now, after the Core has been released?

katjaweigel picture katjaweigel on 21 Jun 2021

For recipe_deangelis15nat.yml: should I remove MIROC5 or is it possible to do the rounding from ESMValGroup/ESMValCore#1110 also for evspsbl and hfls now, after the Core has been released?

Without MIROC5 recipe_deangelis15nat.yml runs.

katjaweigel picture katjaweigel on 21 Jun 2021

For recipe_deangelis15nat.yml: should I remove MIROC5 or is it possible to do the rounding from ESMValGroup/ESMValCore#1110 also for evspsbl and hfls now, after the Core has been released?

I made an issue https://github.com/ESMValGroup/ESMValCore/issues/1191 and a pull request https://github.com/ESMValGroup/ESMValCore/pull/1192 for the Core.

katjaweigel picture katjaweigel on 23 Jun 2021

recipe_perfmetrics_land_CMIP5.yml fails for the fgco2 diagnostics:
```021-06-28 13:04:19,693 UTC [23824] ERROR esmvalcore._main:440 Program terminated abnormally, see stack trace below for more information:
Traceback (most recent call last):
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_main.py", line 433, in run
fire.Fire(ESMValTool())
File "/work/bd0854/b309137/anaconda_20200811/envs/main_v3/lib/python3.9/site-packages/fire/core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/work/bd0854/b309137/anaconda_20200811/envs/main_v3/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/work/bd0854/b309137/anaconda_20200811/envs/main_v3/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace
component = fn(varargs, *kwargs)
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_main.py", line 410, in run
process_recipe(recipe_file=recipe, config_user=cfg)
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_main.py", line 104, in process_recipe
recipe.run()
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_recipe.py", line 1434, in run
self.tasks.run(max_parallel_tasks=self._cfg['max_parallel_tasks'])
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_task.py", line 674, in run
self._run_sequential()
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_task.py", line 685, in _run_sequential
task.run()
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_task.py", line 248, in run
input_files.extend(task.run())
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_task.py", line 248, in run
input_files.extend(task.run())
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/_task.py", line 252, in run
self.output_files = self._run(input_files)
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/__init__.py", line 475, in _run
self.products = _apply_multimodel(self.products, step,
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/__init__.py", line 417, in _apply_multimodel
result = preprocess(products - exclude, step, *settings)
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/__init__.py", line 295, in preprocess
result.append(_run_preproc_function(function, items, settings))
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/__init__.py", line 281, in _run_preproc_function
return function(items, *kwargs)
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 431, in multi_model_statistics
return _multiproduct_statistics(
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 342, in _multiproduct_statistics
statistics_cubes = _multicube_statistics(cubes=cubes,
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 322, in _multicube_statistics
result_cube = _compute_eager(aligned_cubes,
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 277, in _compute_eager
combined_slice = _combine(single_model_slices)
File "/mnt/lustre02/work/bd0854/b309137/v2_split/ESMValCore/esmvalcore/preprocessor/_multimodel.py", line 264, in _combine
merged_cube = cubes.merge_cube()
File "/work/bd0854/b309137/anaconda_20200811/envs/main_v3/lib/python3.9/site-packages/iris/cube.py", line 404, in merge_cube
proto_cube.register(cube, error_on_mismatch=True)
File "/work/bd0854/b309137/anaconda_20200811/envs/main_v3/lib/python3.9/site-packages/iris/_merge.py", line 1361, in register
match = coord_payload.match_signature(
File "/work/bd0854/b309137/anaconda_20200811/envs/main_v3/lib/python3.9/site-packages/iris/_merge.py", line 284, in match_signature
raise iris.exceptions.MergeError(msgs)
iris.exceptions.MergeError: failed to merge into a single cube.
Coordinates in cube.dim_coords differ: longitude.

I was able to track it back to missing bounds in the longitude coordinate for some models - these models go through regridding first. When I added a guess bounds 

    cube.coord("longitude").bounds = None
    cube.coord("longitude").guess_bounds()

```

to longitude in the core _multimodel.py after l.259 it ran through.

Not sure that's the best idea to implement though - but having a problem with the mmm (the recipe ran through before the refactoring of mmm to use native iris https://github.com/ESMValGroup/ESMValCore/pull/1150) seems like a huge issue. Not sure if it'd be better to add (guess) the bounds in general before the other preprocessor steps, or make sure that it's just not a problem in the multi model statistics.

bettina-gier picture bettina-gier on 29 Jun 2021

To keep track of what is still open, I'll mark all comments that have been resolved with a :tada:

zklaus picture zklaus on 1 Jul 2021
🎉1

To help with this task and to prepare for automating it better in the future, I tried to run all recipes on Mistral. The output can be seen here.

Note that if you see a web page when viewing the results of a successful recipe run and some plots or other output is missing from it, this is because provenance has not been implemented in the diagnostic script for these output files. Example of the output of a recipe without provenance: recipe_albedolandcover.yml. Example of a recipe with provenance correctly implemented: examples/recipe_python.yml.

Below is a quick summary of why certain recipes fail, probably duplicating some of the information above.

The runs in June were done with a 1 hour time limit, the runs in July with a 2 hour time limit on the Mistral compute partition. The code for doing the runs is available in #2219.

@ESMValGroup/esmvaltool-developmentteam

Potential bugs

recipe run | reason of failure
---------------|-----------------------
recipe_autoassess_landsurface_soilmoisture_20210625_151027 | recipe uses undocumented auxiliary data, possibly only available on Jasmin
recipe_bock20jgr_20210625_151027 | missing CMIP5 data, recipe needs to change OBS6 to native6 for ERA5, ESMValCore bug in finding CMIP3 data (dates not recognized)
recipe_climwip_test_basic_20210625_151101 | crash in diagnostic script
recipe_clouds_ipcc_20210625_151145 | failed because of crash trying to include too large provenance in PNG file https://github.com/ESMValGroup/ESMValCore/issues/1148
recipe_consecdrydays_20210625_151145 | diagnostic script crashed with TypeError: quickplot() got multiple values for argument 'plot_type'
recipe_deangelis15nat_20210625_151145 | failed to derive lvp because Coordinate 'latitude' has different points for the LHS cube 'precipitation_flux' and RHS cube 'water_evaporation_flux'
recipe_impact_20210625_151714 | seems to fail on CMOR check of AWI-CM-1-1-MR areacella/o
recipe_kcs_20210625_151714 | multimodel statistics crash: cube data dtype differs: float32 != float64
recipe_meehl20sciadv_20210625_151748 | seems to fail on CMOR check of AWI-CM-1-1-MR areacella/o
recipe_ocean_bgc_20210625_152235 | tries to compute multimodel statistics on single cube
recipe_ocean_Landschuetzer2016_20210625_152110 | multimodel statistics crash: cube data dtype differs: float32 != float64
recipe_ocean_multimap_20210625_152327 | cube arithmetic bug or data issue in diagnostic script
recipe_preprocessor_test_20210625_150851 | cube.ancillary_variables differ in multimodel statistics
recipe_seaice_drift_20210625_152519 | diagnostic script crash (data or shapefile issue?)

Computational issues?

recipe run | reason of failure
---------------|-----------------------
recipe_extreme_events_20210625_151332 | processing took more than 1 hr?
recipe_eyring06jgr_20210625_151418 | ran out of memory?
recipe_eyring13jgr_12_20210625_151418 | ran out of memory
recipe_lisflood_20210625_151027 | hangs? ran out of time/memory?
recipe_martin18grl_20210625_151714 | processing took more than 1 hr?
recipe_smpi_20210625_152604/ | ran out of memory during linear regrid of CMIP5 GFDL-ESM2G ta to ERA5 grid
recipe_wenzel16jclim_20210625_152840 | processing took more than 1 hr?

Missing data

recipe run | reason of failure
---------------|-----------------------
recipe_anav13jclim_20210625_151027 | missing CMIP5 data
recipe_check_obs_20210625_150707 | missing OBS data v1.1-CRU+GPCC and v1.1-CRU
recipe_climwip_brunner20esd_20210625_151101 | missing CMIP6 data
recipe_climwip_test_performance_sigma_20210625_151101 | missing CMIP6 data
recipe_collins13ipcc_20210625_151145 | missing CMIP5 data
recipe_ecs_constraints_20210625_151321 | missing ERA5 data
recipe_era5_20210625_150707 | missing ERA5 data
recipe_era5-land_20210625_150707 | missing ERA5 data
recipe_flato13ipcc_20210625_151434 | missing CMIP3, CMIP5 data
recipe_gier2020bg_20210625_151434 | missing OBS, CMIP5, CMIP6 data
recipe_globwat_20210625_151027 | missing ERA5 data
recipe_hydro_forcing_20210625_151027 | missing ERA5 data
recipe_hype_20210625_151027 | missing ERA5 data
recipe_landcover_20210625_151714 | missing CMIP5 data
recipe_marrmot_20210625_151027 | missing ERA5 data
recipe_pcrglobwb_20210625_151027 | missing ERA5 data
recipe_perfmetrics_CMIP5_20210625_152344 | missing CMIP5 data
recipe_perfmetrics_CMIP5_4cds_20210625_152327 | missing CMIP5 data
recipe_perfmetrics_land_CMIP5_20210625_152344 | missing CMIP5 data
recipe_preprocessor_derive_test_20210625_150851 | missing CMIP5 data
recipe_schlund20esd_20210625_152519 | missing CMIP5 data
recipe_schlund20jgr_gpp_abs_rcp85_20210625_152858 | missing CMIP5 data
recipe_schlund20jgr_gpp_change_1pct_20210625_152936 | missing CMIP5 data
recipe_schlund20jgr_gpp_change_rcp85_20210625_152936 | missing CMIP5 data
recipe_seaice_20210625_152519 | missing CMIP5 sic and areacello data
recipe_seaice_feedback_20210625_152519 | missing CMIP5 data
recipe_smpi_4cds_20210625_152604 | missing CMIP5 data
recipe_snowalbedo_20210625_152605 | missing CMIP5 data
recipe_wenzel14jgr_20210625_152752 | missing CMIP5 data for variable nbp
recipe_wenzel16nat_20210625_152840 | missing CMIP5 data MIROC-ESM variable gpp
recipe_wflow_20210625_151027 | missing ERA5 data
recipe_williams09climdyn_CREM_20210625_152840 | missing CMIP5 and CMIP6 data

bouweandela picture bouweandela on 2 Jul 2021
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

bouweandela picture bouweandela  路  3Comments
chris-to-pher picture chris-to-pher  路  3Comments
bascrezee picture bascrezee  路  5Comments
valeriupredoi picture valeriupredoi  路  5Comments
bouweandela picture bouweandela  路  4Comments