Esmvaltool: How do I preprocess World Ocean Atlas (WOA) datasets?

[ledm]

At the Crescendo meeting today, I just showed a multi-model marine oxygen comparison, but didn't include observations. @zklaus said that several tools are available for adding observational datasets, but that these were mostly set up for version 1 of ESMValTool.

I've only worked with Obs4mip observations files before in ESMvalTool. Outside of OBs4Mips, many more marine datasets are available, but I doubt that many are fullly cmor compliant.

How do I add the World Ocean Atlas data into the preprossors? Is there a pre-preprocesor tool for obs?

[zklaus]

Hey Lee,
have a look in the development branch of version 1.
You will actually find a reformatting script for WOA09. There are instructions inside the respective reformatting scripts.

[mattiarighi]

We plan to port the framework for reformatting the obs data to version2 soon.
See #232.

[mattiarighi]

Also note, that the preprocessor in v2 is much stricter than in v1 concerning the metadata. Therefore it could be that the v1 reformatting scripts need some changes to get a v2-compliant dataset.

[ledm]

Are we expected to write the reformatting script using NCL or can we do it in python?

[zklaus]

Can be done in python. I did that for WOCE, if you want to have a look at an example.

[ledm]

Thanks @zklaus, where can I find that?

[zklaus]

Ha, good point. Do you have access to the private repository?
https://github.com/ESMValGroup/ESMValTool-private/blob/development/reformat_scripts/obs/reformat_obs_woce.py

[ledm]

Apparently, I do! Thanks.

[bouweandela]

If you are working in Python, you can also have a look at implementing it as a CIS plugin, see this comment.

[ledm]

Thanks. I think I've managed to cobble together a script from @zklaus's reformat python script. It makes the WOA data CF-compliant enough to be loaded by iris. However, the WOA data are climatological.

• What kind of date information, calendar and time units are expected in an ESMValTool climatology?
• How should you define the time range in an ESMValTool recipe to load a climatology?

[mattiarighi]

Is this a 12-month climatology?
In this case, you can assign the central year as a time coordinate (e.g., year 2000 for a 1990-2010 climatology) and use this a start_year and end_year in the recipe.

[mattiarighi]

Please keep your script aside until #232 is solved and then open a PR to add it to v2 (you can assign it to me).

[ledm]

Good news, I've managed to get the WOA data script into a useable format!

Here are some figures:

Historical (r1i1p1) HadGEM2-ES in 2001-2004 vs WOA at 500m:

Historical (r1i1p1) HadGEM2-ES in 2001-2004 vs WOA at 500m (scatter):

And a multi model comparison of oxygen at 300m with a threshold of .03mol/m-3. Once again, this is the historical scenario in 2001-2004 vs WOA oxygen.

[valeriupredoi]

are you fitting a line through a shark @ledm ? :rofl: well done, this should wait a bit more for #666 then you can test with the new cmorize_observations executable

[ledm]

These plots also need to wait for PR #550 to be merged, as they require irregular grids.

In the meanwhile, I've also extended my tools to cover WOA nitrate:

WOA silicate:

and WOA phosphate:

All these are annual climatologies from the objectively analyzed mean fields at standard depth levels at 500m, between the years 2001 and 2004.

I'd be happy to compare my netcdfs to the results of a cmorise_observables. How does it work, @valeriupredoi?