Rasterio: Rasterio 1.2.0

[sgillies]

Let's do this before the end of October.

New features include:

• VSI cache clearing #1078
• API for driver metadata (like https://github.com/Toblerity/Fiona/pull/950)
• RPC support #1922
• More: https://github.com/mapbox/rasterio/milestone/95?closed=1

[sgillies]

End of October was too optimistic. I need to move the target to end of November. Sorry, everybody.

There are some kind of big issues to address still. Such as

• How to address datasets in the cloud. See for example https://github.com/mapbox/rasterio/pull/1906. I can't abide /vsis3/ &c, but I'm not sure that rasterio is providing a better solution.
• RPCs https://github.com/mapbox/rasterio/pull/1922

[sgillies]

As soon as #2054 is merged, I'm going to tag 1.2a1 and publish some dists to PyPI. Maybe later today. These will be wheels with patched GDAL 2.4.4. There are still some issues to sort out with GDAL 3.2 and PROJ 7.2.

[sgillies]

https://pypi.org/project/rasterio/1.2a1/#files

[sgillies]

Got some positive feedback already. Let's try for a 1.2b1 by 2020-12-15.

[sgillies]

I'm building wheels for 1.2b2 now and will upload to PyPI later tonight or in the morning.

[sgillies]

1.2 :bee: :two: https://pypi.org/project/rasterio/1.2b2/#files

[sgillies]

1.2 :bee: :three: coming up in an hour...

[sgillies]

https://pypi.org/project/rasterio/1.2b3/#files

[snowman2]

Any interest in bumping the GDAL version in the wheels to 3.2.1 and PROJ version to 7.2.1?

[sgillies]

@snowman2 good call! I'll do that for the 1.2b4 wheels I intended to upload this afternoon.

[sgillies]

Welp, GDAL 3.2.1 doesn't build on our wheel infra. Let's deal with this after 1.2.0.

In file included from ogr_elastic.h:33:0,
                 from ogrelasticlayer.cpp:30:
/io/gdal-3.2.1/ogr/ogrsf_frmts/ogrsf_frmts.h: In lambda function:
/io/gdal-3.2.1/ogr/ogrsf_frmts/ogrsf_frmts.h:109:18: error: ‘OGRErr OGRLayer::GetExtentInternal(int, OGREnvelope*, int)’ is protected
     OGRErr       GetExtentInternal(int iGeomField, OGREnvelope *psExtent, int bForce );
                  ^
ogrelasticlayer.cpp:3513:83: error: within this context
         const auto eRet = OGRLayer::GetExtentInternal(iGeomField, psExtent, bForce);

[sgillies]

Reported upstream https://github.com/OSGeo/gdal/issues/3374. Removing the elastic driver support could be another option.

[sgillies]

And as to PROJ 7.2.1, I'm seeing some failures when running tests on the wheels. I think we're going to have to go with 7.2.0.

=================================== FAILURES ===================================
________________________________ test_read_epsg ________________________________
    def test_read_epsg():
        with rasterio.open('tests/data/RGB.byte.tif') as src:
>           assert src.crs.to_epsg() == 32618
E           assert None == 32618
E            +  where None = <bound method CRS.to_epsg of CRS.from_wkt('PROJCS["UTM Zone 18, Northern Hemisphere",GEOGCS["Unknown datum based upon ...,PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH]]')>()
E            +    where <bound method CRS.to_epsg of CRS.from_wkt('PROJCS["UTM Zone 18, Northern Hemisphere",GEOGCS["Unknown datum based upon ...,PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH]]')> = CRS.from_wkt('PROJCS["UTM Zone 18, Northern Hemisphere",GEOGCS["Unknown datum based upon the WGS 84 ellipsoid",DATUM["...],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH]]').to_epsg
E            +      where CRS.from_wkt('PROJCS["UTM Zone 18, Northern Hemisphere",GEOGCS["Unknown datum based upon the WGS 84 ellipsoid",DATUM["...],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH]]') = <open DatasetReader name='tests/data/RGB.byte.tif' mode='r'>.crs
tests/test_crs.py:82: AssertionError

[snowman2]

Well, at least it was able to uncover some issues to direct upstream :upside_down_face:. Thanks for giving it a try :+1:

[snowman2]

@sgillies, it is strange that the issue does not also occur in PROJ 7.2.0. This is the behavior I see in both 7.2.0 and 7.2.1:

>> import pyproj
>>> crs = pyproj.CRS('PROJCS["UTM Zone 18, Northern Hemisphere",GEOGCS["Unknown datum based upon the WGS 84 ellipsoid",DATUM["Not_specified_based_on_WGS_84_spheroid",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-75],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH]]')
>>> crs
<Projected CRS: PROJCS["UTM Zone 18, Northern Hemisphere",GEOGCS[" ...>
Name: UTM Zone 18, Northern Hemisphere
Axis Info [cartesian]:
- [east]: Easting (metre)
- [north]: Northing (metre)
Area of Use:
- undefined
Coordinate Operation:
- name: UTM zone 18N
- method: Transverse Mercator
Datum: Not_specified_based_on_WGS_84_spheroid
- Ellipsoid: WGS 84
- Prime Meridian: Greenwich
>>> crs.to_epsg()
>>> crs.to_epsg(60)
3450
>>> pyproj.__proj_version__
'7.2.0'

[snowman2]

Seems to be more confident in 7,0.1:

>>> crs.to_epsg()
3450
>>> pyproj.proj_version_str
'7.0.1'

[sgillies]

@snowman2 don't be too concerned. It's also quite possible that my trouble is very specific to the rasterio wheels. PROJ 7.2.1 has a new version of the EPSG database, right? It couldn't have removed 32618, that would be surprising.

[snowman2]

PROJ 7.2.1 has a new version of the EPSG database, right? It couldn't have removed 32618, that would be surprising.

It does have a new version of the database, but 32618 is still in there.

>>> pyproj.__proj_version__
'7.2.1'
>>> pyproj.CRS(32618).to_epsg()
32618

[snowman2]

Side note: the to_epsg method picks from the top of a list of potential EPSG codes. So 32618 could have been lower in the list for some reason.

[sgillies]

I think I found the problem in the tests of the built wheels: I had forgotten to unset PROJ_LIB, which isn't needed anymore :facepalm:

[sgillies]

I upgraded PROJ and GDAL on my laptop to 7.2.1 and 3.2.1 and rasterio is broken. It's not just a busted build configuration.

[sgillies]

Okay, I think we're back on track for a 1.2b4 tomorrow morning.

[sgillies]

https://pypi.org/project/rasterio/1.2b4/#files

[sgillies]

Unless we find new bugs, let's plan for a 1.2.0 on 2021-01-25.

[sgillies]

Done! Release note sent to the user group. Thanks, everyone.

[vincentsarago]

👏 Thanks for the hard work @sgillies