Esmvaltool: Beginner's questions

[chris-to-pher]

Hi all,

not reporting a bug, simply looking for help:

I followed the instructions on readthedocs and installed the ESMValTool on mistral at DKRZ. My first attempt was to run the standard example recipe_python.yml with synda. The files are found but the download does not work:
NoOptionError: No option 'destination_directory' in section: 'globustransfer'
ERROR Failed to run download

Do I need to add a directory somewhere in the call of ESMValTool? Or is it a problem with globustransfer (not really sure what that is)? On the synda documentation it says that it's only available by installing from source, however I installed synda using conda as suggested in the ESMValTool documentation.

main_log_debug.txt
main_log.txt

My second attempt was to run the same recipe with input from the local CMIP5 data pool at DKRZ (located at /work/kd0956). The files are found, but then there is the Error "Can not concatenate cubes into a single one." What could be the reason for that?

main_log_debug.txt
main_log.txt

Could you please help me with these 2 (unrelated) questions? I attached all the log files.

Thanks a lot,
Christopher

[valeriupredoi]

Hi @chris-to-pher

• about the synda download, I haven't used that in ages and would not advise using it since you already have direct access to the DKRZ ESGF data node by the looks of it; maybe @bouweandela or @bjoernbroetz have used the download functionality more recently;

• about the concatenation error: that is happening because the code is finding all availbale datasets:

INFO    [9892] Using input files for variable pr of dataset MPI-ESM-LR:
/work/kd0956/CMIP5/data/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20111006/pr/pr_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc
/work/kd0956/CMIP5/data/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20111119/pr/pr_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc
/work/kd0956/CMIP5/data/cmip5/output1/MPI-M/MPI-ESM-LR/historical/mon/atmos/Amon/r1i1p1/v20120315/pr/pr_Amon_MPI-ESM-LR_historical_r1i1p1_185001-200512.nc

regardless of version; this is happening because you have not set the correct DRS in your configuration file. A DRS is needed, eg, at the bottom of your configuration file:

# Directory structure for input data: [default]/BADC/DKRZ/ETHZ/etc
# See config-developer.yml for definitions.
drs:
  CMIP5: DKRZ

so that the data finder picks up the right type of path and correct latest version for the data, avoiding data tree duplication. On the same topic, you should probably set the CMIP5 rootpath to match the correct one on DKRZ:

rootpath:
  # Rootpath to CMIP5 data
  CMIP5: /work/kd0956/CMIP5/data/cmip5/output1

Please have a look at the documentaion https://esmvaltool.readthedocs.io/projects/esmvalcore/en/latest/esmvalcore/datafinder.html Cheers :beer:

[bouweandela]

I followed the instructions on readthedocs and installed the ESMValTool on mistral at DKRZ. My first attempt was to run the standard example recipe_python.yml with synda. The files are found but the download does not work:
NoOptionError: No option 'destination_directory' in section: 'globustransfer'
ERROR Failed to run download

Do I need to add a directory somewhere in the call of ESMValTool? Or is it a problem with globustransfer (not really sure what that is)? On the synda documentation it says that it's only available by installing from source, however I installed synda using conda as suggested in the ESMValTool documentation.

It looks like your synda configuration might not be correct, this problem is not related to ESMValTool. As @valeriupredoi mentioned, I would recommend using the data you already have locally, but if you really want to download using synda, I would recommend having a look at the synda documentation: http://prodiguer.github.io/synda/sdt/conda_install.html

maybe @bouweandela or @bjoernbroetz have used the download functionality more recently

I haved used it recently and it works mostly fine, provided that synda is installed and configured correctly.

[chris-to-pher]

Thanks for your help. Running the example with the local data at DKRZ works now (and I will not worry about the synda problem for now).