FionaValueError: No dataset found at path

[vanderlist]

I suddenly keep getting:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\fiona\__init__.py", line 175, in open
    enabled_drivers=enabled_drivers)
  File "C:\Python27\lib\site-packages\fiona\collection.py", line 147, in __init__
    self.session.start(self)
  File "fiona\ogrext.pyx", line 444, in fiona.ogrext.Session.start (fiona/ogrext.c:9253)
FionaValueError: No dataset found at path '\test.shp' using drivers: *

I use fiona through rasterstats (https://github.com/perrygeo/python-rasterstats). I get this with every .shp files. Even those that used to work.

The problem is that I don't know exactly what changed.
I've updated to:
Python 2.7.13
And from Gohlke:
Fiona 1.7.1
GDAL 2.0.2 (because the 2.1.2 version resulted in Python crashes)
From gisinternals
GDAL 2.1.2
and it started not working.

I used to use:
Python 2.7.11
And from Gohlke:
Fiona 1.6.3
GDAL 2.0.2
From gisinternals
GDAL 2.1.0

The problem is when I uninstall everything (even GDAL from gisinternals) and go back to 1.6.3, the problem persists.
ogrext.pyx doesn't seems to be anywhere (only .pyd) but I guess that's normal?

I apologize already for probably not giving enough information. Many thanks in advance.

[sgillies]

@vanderlist can you provide a little more context, like a complete script (smaller the better) that produces the error? I'll try it out.

[vanderlist]

That's the thing. Just a basic operation invokes it:

with rasterstats:

from rasterstats import zonal_stats
oStats = zonal_stats('test.shp', 'test.tif')

and just with fiona:

import fiona 
c = fiona.open('test.shp', 'r')

I first thought it was a SHP issue but it happens too with SHP files that worked before. So I guess you can use whatever SHP file you got. For rasterstats you can use the test data from the package itself.

Then I thought it was a rasterstats issue so I went from 0.11.0 back to 0.10.3, same problem.
Then I tried reinstalling GDAL 2.1.0 back instead of GDAL 2.1.2 from gisinternals.
Then I tried reinstalling everything including Python.

If there is anything else I can provide. I'm working on Windows 10 using Visual Studio 2015 and the Python Tools: PTVS 2.2.50104.00-14.0.
a full list of the python packages

I had rasterio 0.33.0 and pyshp 1.2.3 before but I don't think it matters.

The only similar problems on the internet, I can find is the problem Fiona _had_ with GDAL2.0 instead of GDAL1.0.
I really hope you can reproduce but I doubt it. I hope I'm overseeing something somewhere.
Many thanks in advance for looking into it.

[rbuffat]

I suspect that there is something wrong with the gdal installation you have. I'm not sure if you can mix gdal from gisinternals with the fiona binaries from gohlke. If you use the fiona binaries, I would suggest to use the same gdal libraries fiona is compiled against. Do you have other versions of gdal installed, e.g. from osgeo4w? Could you check if ogrinfo works? It should be located in C:PythonLibsite-packagesosgeo or similar.

Regarding orgext.pyx: Gdal 1.x ist not API compatible to Gdal 2.x. Fiona uses orgext1.pyx for gdal 1.x and ogrext2.pyx for gdal 2.x. At compile time, ogrext.pyx is created from either orgext1.pyx or orgext2.pyx depending on the installed version of gdal.

[vanderlist]

Hello,

I seem to have a solution for fiona. So I tried what @rbuffat suggested and deinstalled everything and not reinstalled the GISinternals GDAL. I even deinstalled QGIS to have no other Python and/or GDAL where ever. Although it had worked before with them installed.

I then installed my previous setup again.
Python 2.7.11
And from Gohlke:
Fiona 1.6.3 (Fiona-1.6.3+gdal200-cp27-none-win_amd64.whl)
GDAL 2.0.2 (GDAL-2.0.2-cp27-none-win_amd64.whl)

And added C:Python27Libsite-packagesosgeo as PATH instead of C:Program FilesGDAL so I could use GDAL in cmd (the reason I thought I needed the GISinternals GDAL in the first place). I also changed the GDAL_DATA and GDAL_DRIVER_PATH variables accordingly.

Fiona seems to work now with:

import fiona 
c = fiona.open('test.shp', 'r')

However rasterstats 0.11.0 does not. With:

from rasterstats import zonal_stats
oStats = zonal_stats('test.shp', 'test.tif')

It gives me

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\rasterstats\main.py", line 28, in zonal_stats
    return list(gen_zonal_stats(*args, **kwargs))
  File "C:\Python27\lib\site-packages\rasterstats\main.py", line 140, in gen_zonal_stats
    for _, feat in enumerate(features_iter):
  File "C:\Python27\lib\site-packages\rasterstats\io.py", line 84, in fiona_generator
    for feature in src:
  File "fiona/ogrext.pyx", line 1126, in fiona.ogrext.Iterator.__next__ (fiona/ogrext.c:19067)
  File "fiona/ogrext.pyx", line 164, in fiona.ogrext.FeatureBuilder.build (fiona/ogrext.c:3357)
IndexError: list index out of range

I don't know what goes wrong there.

The problem was indeed the GDAL fiona is looking for when installing.
I also tried this with the updated setup with Python 2.7.13 and Fiona 1.7.1 but that still didn't work. Updating is a problem to deal with later (my problem of course).

One step closer but not yet there.
For the record: ogrinfo works (from C:Python27Libsite-packagesosgeo)

[sgillies]

@vanderlist it is indeed very easy to get into a bad situation with too many GDALs installed. I'm sorry about that!

There's a chance that upgrading to rasterstats 0.10.1 will fix your problem. I see it has a possibly related change: https://github.com/perrygeo/python-rasterstats/commit/c6c85685275f39fa242a96ef7e74889756f2fca7 /cc @perrygeo

[rbuffat]

@sgillies
Would it make sense to implement a fiona.debug() / check() function that checks if the current version of gdal matches the version fiona was compiled against, issuing a warning if they do not match?

[vanderlist]

Hello @sgillies
I was already busy trying out older versions of rasterstats but the problem persists with 0.10.3 and 0.10.1 (so I had to downgrade since it was there in 0.11.0).
Anyway thanks for the suggestion. Should I open an issue at https://github.com/perrygeo/python-rasterstats/issues or is this still fiona related? /cc @perrygeo

Anyway, from a first time poster (but longer user): Thank you all for looking into this.

[perrygeo]

@vanderlist can you try this and report the results:

import fiona
with fiona.open('test.shp', 'r') as src:
    features = [f for f in src]
print(len(features))

If you do only c = fiona.open('test.shp', 'r') it does not iterate over the features which appears to be where the problem occurs.

[vanderlist]

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "fiona/ogrext.pyx", line 1126, in fiona.ogrext.Iterator.__next__ (fiona/ogrext.c:19067)
  File "fiona/ogrext.pyx", line 164, in fiona.ogrext.FeatureBuilder.build (fiona/ogrext.c:3357)
IndexError: list index out of range

You are correct, the same problem. So, it is still a Fiona problem, or more specifically a 'Fiona 1.6.3 on my setup' problem. I will try and upgrade to Fiona 1.7.1 using Gohlke's whl. And see if the problem persists. No use in trying to solve something in an old version, I think.

For the record:
print(len(features)) gives Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'features' is not defined ofc.

[perrygeo]

@vanderlist are you able to share the shapefile? I'd like to rule out a problem in the dataset itself.

[vanderlist]

If I upgrade to Fiona 1.7.1 I get the following with import fiona:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\fiona\__init__.py", line 76, in <module>
    from fiona.collection import Collection, BytesCollection, vsi_path
  File "C:\Python27\lib\site-packages\fiona\collection.py", line 7, in <module>
    from fiona import compat
ImportError: cannot import name compat

I can share the shape file but it happens with all the shape files

sample_selector_grid100_m0_tmp.zip

.

[rbuffat]

I think fiona 1.6.3 is incompatible with gdal 2.0. Support for Gdal 2.x was introduced with 1.7.0.

IndexError: list index out of range Indicates, that the shapefile contains e.g. an Integer64, which Fiona before 1.7.0 is not aware of.

[vanderlist]

Indicates, that the shapefile contains a Integer64, which Fiona before 1.7.0 is not aware of.

This is the case. But it worked before. From Golhke:

• Fiona-1.6.3+gdal200-cp27-none-win_amd64.whl
• GDAL-2.0.2-cp27-none-win_amd64.whl

Of course I did not know which GDAL Fiona used (Gohlke's or GisInternals GDAL, both were 2.x though).
Anyway, strange. Maybe Gohlke did something in the .whl to make it work (hence the gdal200 reference in filename)?

Sorry for not understanding. I think I'm really overlooking something that happened and broke it. I will look into the compat ImportError.

[vanderlist]

Correction:
With Fiona 1.7.1. With import fiona I get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\fiona\__init__.py", line 76, in <module>
    from fiona.collection import Collection, BytesCollection, vsi_path
  File "C:\Python27\lib\site-packages\fiona\collection.py", line 8, in <module>
    from fiona.ogrext import Iterator, ItemsIterator, KeysIterator
ImportError: DLL load failed: The specified module could not be found.

It's only after I try to from rasterstats import zonal_stats that it get:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\fiona\__init__.py", line 76, in <module>
    from fiona.collection import Collection, BytesCollection, vsi_path
  File "C:\Python27\lib\site-packages\fiona\collection.py", line 7, in <module>
    from fiona import compat
ImportError: cannot import name compat

[vanderlist]

Ok, it's defintely a dependency problem with Fiona 1.7.1 not finding the GDAL dll. I checked ogrext.pyd with Dependendacy Walker. It searches for GDAL201.dll which it does not find in the osgeo folder because it has GDAL200.dll . If try the same with Fiona 1.6.3 it searches for GDAL200.dll .
Is there a way I can force Fiona 1.7.1 to look for GDAL200.dll ?
Does pip install do something wrong when installing?
tbc.

[rbuffat]

I would recommend deinstalling all gdal / fiona / rasterio libraries.Then check site-packages for left-overs. Then download the latest gdal / fiona / rasterio wheels from gohlke and install them. If you sue the wheels from gohlke, I would only use wheels that are downloaded at the same time, because of the dependencies with gdal.

[vanderlist]

Good news everyone, I got it working.
Even though I tried reinstalling everything several times I got it working with installing after deinstalling the packages on Python 2.7.11:

• rasterio-1.0a3-cp27-cp27m-win_amd64.whl
• GDAL-2.1.2-cp27-cp27m-win_amd64.whl
• Fiona-1.7.1-cp27-cp27m-win_amd64.whl
• rasterstats 0.11.0

To recapitulate:
The original Fiona error I got probably started when I updated the GISinternals GDAL (from 2.1.0 to 2.1.2). I didn't uninstall it first so it probably also got faulty. While this still being installed I updated the Gohlke GDAL from 2.0.2 to 2.1.2 and this made python crash so I went back to Gohlke GDAL 2.0.2. And that didn't match with Fiona 1.7.1. So I went back to Fiona 1.6.3 and that gave the Indexerror.

Now that the GISinternals GDAL is removed, the Gohlke GDAL 2.1.2 doesn't crash anymore and it now all works but it took some time for me to try this version again. Which was actually the 2nd part of the issue here. I also got a small hiccup went l installed rasterio 0.36.0, but it of course needs Gohlke GDAL 2.0.2 so rasterio1.0a3 did the trick. I can also confirm that rasterstats 0.11.0 is needed (0.10.3 does not work).

It all seems logic now. I would like to thank you, @rbuffat @sgillies and @perrygeo, for dealing with this problem even though you shouldn't have. Thank you for being my listening board.

What still remains (and probably forever):
How did it work with Fiona 1.6.3 and GDAL2.0.2 before I updated? (rhetorical....)

Ciao.

[rbuffat]

Maybe Fiona 1.6.3 from Gohlke was patched with fixes from the fiona 1.7.0 dev branch to work with gdal 2.0. The filename Fiona-1.6.3+gdal200-cp27-none-win32.whl suggests something like this.