Esmvaltool: AURA-TES cmorizer not running at DKRZ

[mattiarighi]

As reported and discussed in #1163, for some strange reason the AURA-TES cmorizer is not running at DKRZ.

Apparently the NCL script is not executing, although all other cmorizers work fine and produce output.

It would be good if some of the other ESMValToolers at DKRZ (@schlunma @axel-lauer @hb326) could try it to rule out installation problem from my side:

cmorize_obs -c config-user.yml -o AURA-TES

(mind the uppercase).

This suggestion by @valeriupredoi should also be tried.

[bouweandela]

Could you try running the script directly from you shell? i.e. first run the cmorize_obs command, and after it failed, set the required environmental variables manually and run ncl /path/to/aura_tes_cmorizer.ncl?

[schlunma]

Tested and got similar results as @mattiarighi: AURA-TES runs through without creating outputs. However, other NCL CMORizers stop at

Executing cmd: ncl ~/ESMValTool_2/esmvaltool/cmorizers/obs/cmorize_obs_ncep.ncl

I waited about 10min. Unfortunately I cannot perform further tests right now as I will only return to my computer at about 6pm.

[valeriupredoi]

Tested and got similar results as @mattiarighi: AURA-TES runs through without creating outputs. However, other NCL CMORizers stop at

Executing cmd: ncl ~/ESMValTool_2/esmvaltool/cmorizers/obs/cmorize_obs_ncep.ncl

I waited about 10min. Unfortunately I cannot perform further tests right now as I will only return to my computer at about 6pm.

there is a specific problem with your platform that prevents starting up NCL from Python in your shell. You need to make the distinction between NCL actually starting up and running and it not starting at all. I suggest, as both Bouwe and myself said, run the cmorizer straight as a command line invocation to NCL and then test the launching of NCL from Python using the script from https://github.com/ESMValGroup/ESMValTool/pull/1163#issuecomment-505394315

Manu, also careful with squiggles ie if your std output says Executing cmd: ncl ~/ESMValTool_2/esmvaltool/cmorizers/obs/cmorize_obs_ncep.ncl and not Executing cmd: ncl /home/$USER/$whatevs/ESMValTool_2/esmvaltool/cmorizers/obs/cmorize_obs_ncep.ncl it's not ideal because that path should always be a fully formatted path :beer:

[mattiarighi]

But why only for aura-tes?
Era-interim, ncep, Patmos...all others are running.

[schlunma]

Manu, also careful with squiggles ie if your std output says Executing cmd: ncl ~/ESMValTool_2/esmvaltool/cmorizers/obs/cmorize_obs_ncep.ncl and not Executing cmd: ncl /home/$USER/$whatevs/ESMValTool_2/esmvaltool/cmorizers/obs/cmorize_obs_ncep.ncl it's not ideal because that path should always be a fully formatted path 🍺

I changed that for GitHub :wink:

[valeriupredoi]

this still not working for you DKRZ folks?

[mattiarighi]

This is due to a NCL issue with .he5 data, see
https://www.ncl.ucar.edu/Support/talk_archives/2010/2310.html

Basically, opening the input file with addfile produces a Segmentation fault.

I think the only way out is to rewrite the cmorizer in Python.

[mattiarighi]

Actually, it is possible to convert 'he5' to 'nc' using 'ncks'.
I implemented the fix in #1766.