Esmvaltool: Predefined set of levels for vertical regridding

[mattiarighi]

For some applications, it is necessary to have all models on the same vertical coordinates.

In the current implementation (@bjlittle and @valeriupredoi please correct me if I'm wrong), it is possible to specify in the namelist one or more target levels for the regridding.

It might be useful, however, to implement some sets of predefined levels, that the user can refer to using a tag in select_level. One example could be the CMIP5 standard levels as defined in the CMOR tables (100000. 92500. 85000. 70000. 60000. 50000. 40000. 30000. 25000. 20000. 15000. 10000. 7000. 5000. 3000. 2000. 1000. Pa).

Those predefined levels shall be coded somewhere in the backend code (e.g. in regrid.py) and could be referred to in the namelist in this way:

PREPROCESS:
- {id: pp1, select_level: {levels: CMIP5_levels, scheme: linear}, ...}

Suggestions are welcome!

[valeriupredoi]

Hi Mattia,

Yes, as it is now, one can select a set of vertical levels by initializing
a list in levels: [list] populated with however many levels one needs --
the only problem with this is that if there are too many levels the name of
the intermediary file after select level (if any at all, depending on
config option) will be huge. I will wait until people have decided on this
proposal and code it up, it's an easy-peasy code change.

Cheers,
V

On Fri, Nov 17, 2017 at 10:43 AM, Mattia Righi [e-mail]
wrote:

For some applications, it is necessary to have all models on the same
vertical coordinates.

In the current implementation (@bjlittle https://github.com/bjlittle
and @valeriupredoi https://github.com/valeriupredoi please correct me
if I'm wrong), it is possible to specify in the namelist one or more target
levels for the regridding.

It might be useful, however, to implement some sets of predefined levels,
that the user can refer to using a tag in select_level. One example could
be the CMIP5 standard levels as defined in the CMOR tables (100000. 92500.

1. 
   
   1. 
      
      
      1. 
         
         
         
         1. 
            
            
            
            
            1. 
               
               
               
               
               
               1. 
                  
                  
                  
                  
                  
                  
                  1. 
                     
                     
                     
                     
                     
                     
                     
                     1. 
                        
                        
                        
                        
                        
                        
                        
                        
                        1. 
                           
                           
                           
                           
                           
                           
                           
                           
                           
                           1. 7000.
                           
                           
                           
                           
                           
                           
                           
                           
                           
                           
                        
                        
                        
                        
                        
                        
                        
                        
                        
                     
                     
                     
                     
                     
                     
                     
                     
                  
                  
                  
                  
                  
                  
                  
               
               
               
               
               
               
            
            
            
            
            
         
         
         
         
      
      
      
   
   
2. 
   
   1. 
      
      
      1. 
         
         
         
         1. Pa).
         
         
         
         
      
      
      
   
   

Those predefined levels shall be coded somewhere in the backend code (e.g.
in regrid.py) and could be referred to in the namelist in this way:

PREPROCESS:

• {id: pp1, select_level: {levels: CMIP5_levels, scheme: linear}, ...}

Suggestions are welcome!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/ESMValGroup/ESMValTool/issues/157, or mute the thread
https://github.com/notifications/unsubscribe-auth/AbpCo-ljTLyN7XILUMCPU48GrSgNS45pks5s3WM_gaJpZM4Qh2pU
.

--
Dr. Valeriu Predoi
Computational Scientist for UKESM Core Team
Department of Meteorology, University of Reading
Earley Gate, Office 1U08
READING, RG6 6BB
United Kingdom
Mobile number: 07847416092

"If one day you be questioning your ability to come up with professional
results, think of this: Noah's ark was built by farmers whereas the Titanic
was crafted by skilled engineers"

[mattiarighi]

@jvegasbsc @bouweandela @nielsdrost @axel-lauer Any suggestion on where to put the definition of these predefined sets in the code?
In the old version we had a file containing all such constants.
Is there anything similar in the current version (see also #31

[jvegasbsc]

There are some predefined sets in the cmor tables. You can see some examples in _plevs_ axis entry in _esmvaltool/interface_scripts/cmip5-cmor-tables/Tables/CMIP5_Amon_ and multiple _plev??_ entries in _esmvaltool/interface_scripts/cmip6-cmor-tables/Tables/CMIP6_coordinate.json_

I think we should support them by reading their definitions directly. We can refer to them in the namelist in the way CMIP5_Amon_plevs or CMIP6_plev7 (I would omit the coordinate table in the naming convention just to make it shorter). This will also make easier to support levels defined by other projects: we will just need the tables in a format we can read.

See

[mattiarighi]

If we can stick to the CMOR definitions that would be a great solution.

[mattiarighi]

Sorry for reopening this @jvegasbsc

I was wondering whether it is possible to use the plev coordinate from a given model as a target grid for vertical regridding, similar to what is happening for the horizontal regridding.

Something like:
```
extract_levels:
levels: ERA-Interim
scheme: linear
````

I'm currently implementing the zonal-plot option of perfmetrics_main and realized that this option would be very useful.

If it's an easy implementation, you can commit it directly in this branch. Thank you!

[jvegasbsc]

I agrre that this is really useful, in fact it will probably be the most usual case

It is a bit more complicated because we need to read it from a sample file instead of just using the values in the tables, but definitely possible. I would add the variable we want to use to keep the syntax consistent with the other case.

extract_levels:
  levels: ERA-Interim_plev
  scheme: linear

I will tell you when it's ready

[bouweandela]

The easiest way to implement this is probably to pass the name of the file containing the required levels to the vinterp function, identical to the way it is implemented for the regrid function. The conversion step model name -> filename is done by the namelist parser in namelist.py.

[jvegasbsc]

Thanks! I will take a look and adapt it to this case.

[valeriupredoi]

careful if you pass levels as string, I've just forced the result cube
writing of 'levels' to be doubles:
https://github.com/ESMValGroup/ESMValTool/commit/fb472997f3530b905fb09f16be3a3da4d7e68cfa

On Fri, Jan 26, 2018 at 12:12 PM, Javier Vegas-Regidor <
[email protected]> wrote:

Thanks! I will take a look and adapt it to this case.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/ESMValGroup/ESMValTool/issues/157#issuecomment-360769520,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AbpCo8HcvbRIeJFR4_0RHBv7yRzd8WOFks5tOcEYgaJpZM4Qh2pU
.

--
Dr. Valeriu Predoi
Computational Scientist for UKESM Core Team
Department of Meteorology, University of Reading
Earley Gate, Office 1U08
READING, RG6 6BB
United Kingdom
Mobile number: 07847416092

"If one day you be questioning your ability to come up with professional
results, think of this: Noah's ark was built by farmers whereas the Titanic
was crafted by skilled engineers"

[jvegasbsc]

I am translating the strings on the fly and passing the interpolator a list of floats

[bouweandela]

@jvegasbsc PR #197 broke namelist_MyVar.yml, can you have a look?
My run ends with the following exception:esmvaltool.interface_scripts.cmor_check.CMORCheckError: plev: is not decreasing on the first model in the list.

[mattiarighi]

@jvegasbsc is this implemented?

[jvegasbsc]

Using models not, I forgot about this feature. I am starting to do it

[jvegasbsc]

I have added the functionality to read the levels from a MODEL_VAR_COORDINATE reference. It is in branch REFACTORING_vertical_levels_from_reference. The only thing missing is that branch is the code to get the reference file, that I think should be easier for you or @bouweandela.

Can you take a look?

[bouweandela]

@jvegasbsc I've updated the branch so this works. I've restored namelist_MyVar.yml to the previous pressure level though, because the NCL diagnostic script in it expects a single pressure level to be selected, not multiple. You can test the new pressure level selection features with namelist_preprocessor_test.yml.

The resulting data still does not pass the CMOR check though, see my comment of January 26.

[jvegasbsc]

I have an automatic fix pending for axis that are not going in the correct direction, preferably in a way that do not implies loading the data. Do not worry to much about this. In fact, I think we should merge this and open a specific issue for this check

[jvegasbsc]

Sorry, the automatic fix is already added. The problem here is that we are running the checks with automatic_fixes=False

[bouweandela]

The checks are indeed running with automatic_fixes=False, because we've already done the fix with the function fix_data.

[mattiarighi]

Thank you for adding this extra option for the vertical levels! :+1:
I will open a PR so we can close this long-standing issue.

[bouweandela]

I've had a closer look at the issue I described above: the problem is that when you select a target set of pressure levels from a CMOR table, e.g. coordinate alt40 from CMIP6 table, that have a stored_direction different from the model data, that model will fail the CMOR check, because the preprocessor applies fixes before it selects pressure level(s).

The cleanest solution would probably be to sort the target levels according to the variable pressure coordinate definition in the function _update_target_levels in namelist.py, but I'm not sure how to identify which coordinate is the pressure level. Do you have any ideas on how to solve this @jvegasbsc ?

[mattiarighi]

The stored_direction should be defined by the positive attribute in the CMOR table (= up or down), but I think that the CMOR check does not take this into account at the moment and always assumes down, which is OK for pressure levels but not for altitude levels (like alt40).

[bouweandela]

Ok. Thanks for pointing that out, I had completely overlooked that alt40 are not pressure levels, so using them with pressure level data is meaningless anyway. I've changed the test namelist so it uses a set of pressure levels instead.

I guess we can address the issue with the stored_direction later then, when we actually have namelists that use something other than pressure levels.