Esmvaltool: Loading scalar cubes not possible in iris < 2

[schlunma]

Another argument for switching to iris > 2 as soon as possible (#451):

Consider the following recipe:

...
preprocessors:
  sp50: &sp50
    extract_levels:
      levels: 5000
      scheme: linear
    extract_region:
      start_latitude: -90
      end_latitude: -60
      start_longitude: 0
      end_longitude: 360
    average_region:
      coord1: latitude
      coord2: longitude

diagnostics:
  diag_ta_sp50:
    variables:
      ta: &variable_settings
        project: CMIP5
        mip: Amon
        field: T3M
        exp: historical
        start_year: 1979
        end_year: 2005
        preprocessor: mean_sp50
    scripts:
...

This fails with the following error:

...
  File "ESMValTool/esmvaltool/preprocessor/__init__.py", line 240, in preprocess
    result = [function(items, **args)]
  File "ESMValTool/esmvaltool/preprocessor/_io.py", line 211, in extract_metadata
    cube = iris.load_cube(filename)
  File "miniconda3/envs/esmvaltool/lib/python3.6/site-packages/iris/__init__.py", line 378, in load_cube
    raise iris.exceptions.ConstraintMismatchError('no cubes found')
iris.exceptions.ConstraintMismatchError: no cubes found

The preprocessed file looks like this:

```netcdf CMIP5_ACCESS1-0_Amon_historical_r1i1p1_T3M_ta_1979-2005 {
dimensions:
bnds = 2 ;
variables:
double ta ;
ta:standard_name = "air_temperature" ;
ta:long_name = "Air Temperature" ;
ta:units = "K" ;
ta:cell_methods = "time: mean latitude: longitude: mean time: mean" ;
ta:coordinates = "day_of_month day_of_year lat lon month_number plev time year" ;
double plev ;
plev:bounds = "plev_bnds" ;
plev:units = "Pa" ;
plev:standard_name = "air_pressure" ;
plev:long_name = "pressure" ;
plev:positive = "down" ;
double plev_bnds(bnds) ;
int64 day_of_month ;
day_of_month:bounds = "day_of_month_bnds" ;
day_of_month:units = "1" ;
day_of_month:long_name = "day_of_month" ;
int64 day_of_month_bnds(bnds) ;
int64 day_of_year ;
day_of_year:bounds = "day_of_year_bnds" ;
day_of_year:units = "1" ;
day_of_year:long_name = "day_of_year" ;
int64 day_of_year_bnds(bnds) ;
double lat ;
lat:bounds = "lat_bnds" ;
lat:units = "degrees_north" ;
lat:standard_name = "latitude" ;
lat:long_name = "latitude" ;
double lat_bnds(bnds) ;
double lon ;
lon:bounds = "lon_bnds" ;
lon:units = "degrees_east" ;
lon:standard_name = "longitude" ;
lon:long_name = "longitude" ;
double lon_bnds(bnds) ;
int64 month_number ;
month_number:bounds = "month_number_bnds" ;
month_number:units = "1" ;
month_number:long_name = "month_number" ;
int64 month_number_bnds(bnds) ;
double time ;
time:bounds = "time_bnds" ;
time:units = "day since 1950-01-01 00:00:00.0000000" ;
time:standard_name = "time" ;
time:long_name = "time" ;
time:calendar = "proleptic_gregorian" ;
double time_bnds(bnds) ;
int64 year ;
year:bounds = "year_bnds" ;
year:units = "1" ;
year:long_name = "year" ;
int64 year_bnds(bnds) ;

// global attributes:
...
data:

ta = 228.835393798659 ;

plev = 25000 ;

plev_bnds = 27500, 22500 ;

day_of_month = 15 ;

day_of_month_bnds = 15, 16 ;

day_of_year = 183 ;

day_of_year_bnds = 16, 351 ;

lat = 0 ;

lat_bnds = -31.25, 31.25 ;

lon = 180 ;

lon_bnds = 0, 360 ;

month_number = 6 ;

month_number_bnds = 1, 12 ;

time = 15523 ;

time_bnds = 10592, 20454 ;

year = 1992 ;

year_bnds = 1979, 2005 ;
}
```

The cube is scalar and has no dimensional coordinate left. The current iris version is not able to load files like this, version 2.2.0 is. Right now, this is really inconvenient, since I cannot use the full capacity of the preprocessor and have to do temporal averaging in the diagnostic itself.

[valeriupredoi]

do we have any solid reason why not switching to 2.2.0 like now? @bouweandela @jvegasbsc @mattiarighi

[mattiarighi]

The increase in CPU time and memory with Iris 2 is my biggest worry.

The slow execution times / large memory usage was the main driver for rewriting the ESMValTool
backend. We have been successful in that sense, since now we are factor 10 to 20 faster (depending on the recipe).

We have to make sure that switching to Iris 2 does not make one of our main achievements useless.

Is there a branch when we can run tests for some selected recipes and get some actual numbers for the discussion (to be continued in #451).

[valeriupredoi]

just opened a can of worms: no, there is no such thing, but I am eager to start it :snake: let's have a talk about it next week!

[mattiarighi]

If you could create a branch and make the necessary changes in the code so that it runs with Iris 2 I'll be happy to run some tests, so we can have some figures for the discussion next week. :beer:

[valeriupredoi]

@mattiarighi Ill do that tomorrow - code runs fine with iris 2.2.0 without any changes (well, apart from the one in environment.yml but you can just install with the default 1.13 and then supersede it with 2.2.0 in the virtual environment ie conda install -c conda-forge iris=2.2.0)

[mattiarighi]

Ok, but let's have a branch in case we need to optimize something.

[valeriupredoi]

will close via #832 (when the case)