Shapely: Kernel death on Shapely call deep in a cartopy call stack

[ResidentMario]

Background

I'm working on a geographic plotting library at the moment. The backend is based on cartopy, an extension to matplotlib which modifies matplotlib for the purposes of defining and working with cartographic projections.

cartopy, in turn, makes extensive use of shapely internally.

One particular augmentation that cartopy makes is providing a set_extent method for cartopy.GeoAxes objects (the cartopy drop-in replacement for matplotlib.Axes). I am having trouble, however, with the method: it keeps causing kernel deaths every time the statement is executed.

Trace

I spun up pdb and poked down to see what was causing it. The call stack is huge, but seems to resolve around a particular shapely.geometry.Polygon.__geom__ call deep in the sequence.

I've flattened the failure into a script execution, with the pdb session attached, here. You can reproduce the issue yourself by cloning the repo and executing the runtime-failure.ipynb Jupyter notebook.

The kernel dies as a result of a failure on the C side of Shapely's execution. The following logline is printed to the Jupyter runtime console:

Assertion failed: (0 != cs), function GEOSCoordSeq_getSize_r, file geos_ts_c.cpp, line 3797.

Since I am not a C author I cannot debug this any further than that...

Platform

At the moment I can only confirm that this error occurs on my Mac Air OSX El Capitan. brew list says I have geos: stable 3.5.0 installed, pip says I have geos 3.5.0 and Shapely 1.5.16.

To-do
I will try to simplify the minimal failing example and test this out on my Windows as well, as that platform was having similar issues.

[QuLogic]

Try pip install --no-binary shapely and see #416.

[sgillies]

Yes, that is the best thing to do. Thanks for the help, @QuLogic!

[ResidentMario]

I successfully installed shapely using pip install --no-binary shapely shapely. However, now import cartopy.crs as ccrs fails with the following error:

Invalid argument (must be a Polygon)
Geometry must be a Point or LineString

The console prints the same C call failure:

Assertion failed: (0 != cs), function GEOSCoordSeq_getSize_r, file geos_ts_c.cpp, line 3797.

[ResidentMario]

On Windows I am having a separate but related problem: fiona fails internally (and doesn't even seem to raise a flag while doing so) when attempting to read via import geopandas as gpd; gpd.read_file("<FILENAME>", driver='GeoJSON'). I suspect that the root cause is the same.

Unfortunately it appears I have no recourse with regards to further development until this particular dependency hell gets sorted out. Boo.

[sgillies]

@ResidentMario I tried cloning your repo but it is incomplete, missing data, doesn't completely specify its dependencies. I can't use it to debug. I'm pretty sure that incompatible GEOS DLLs from different sources (Conda, Homebrew, others) are the root of your problem, not Shapely.

[ResidentMario]

Apologies, I had not set up the repository with expectation that it would be used for debugging.

Windows

The following sequence of actions should be a minimal reproducible failure on Windows, tested using a completely clean Ananconda 3.5 install:

1. Get on a Windows machine (obviously).
2. conda create --name geoplot pandas jupyter requests
3. git clone the geoplot repository, navigate to the /dependencies folder and install all of the wheels---for cartopy, fiona, gdal, pyroj, and shapely, specifically. These wheels are the latest 64-bit versions taken from the unofficial directory.
4. pip install geopandas
5. Run jupyter notebook, navigate to runtime-failure-two.ipynb, and execute it.

This results in a kernel failure on my machine.

The computer in question does not have GEOS or GDAL installed through any other source.

Mac OSX

Still working on this one.

[QuLogic]

Why are you installing wheels if you're using conda? Mixing binaries is probably rather fragile. If they're not in defaults, you should be able to get all these packages from the conda-forge channel.

[ResidentMario]

When I first ran into a dependency installation issue many months ago, I Googled for a solution, and using that was the first thing that came up that worked. It's always worked for me ever since then (until now, that is), and though I was aware of conda-forge this is the first time I've heard that it's better for this. Not surprising, but still, the first time I'm hearing it.

I'm away from that machine now, but I will try the conda-forge approach tomorrow.

[QuLogic]

Well, I don't use Windows, so I can't say which is better, but it seems like using a consistent source for everything is _probably_ less likely to be a problem.

[ResidentMario]

To paraphrase from the bug report linked to above:

I believe that running brew install gdal instead gets around this specific issue, insofar as it installs a version of libproj.dylib which gets linked to cartopy correctly.

However, installing proj in that manner installs much of the rest of the C geospatial software bear, which I believe may cause dependency conflicts deeper into the stack.

...which is the issue I saw and reported here.

[sgillies]

See also #553.