Geopandas: BUG: geopandas.read_file only works for zipped shapefiles on disk. (Not streams/open calls)

[cjshowalter]

• [X] I have checked that this issue has not already been reported.

• [X] I have confirmed this bug exists on the latest version of geopandas.

• [ ] (optional) I have confirmed this bug exists on the master branch of geopandas.

---

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

import geopandas
# Works great if reading shapefile set from disk directly:
geopandas.read_file("zip://UTM_7N.zip")

```python
import geopandas

With loading this first:

file = open("UTM_7N.zip", "rb") # Without 'rb', it fails to read.

Can't get the shapefile set to read correctly:

geopandas.read_file(file)

```python
import geopandas
# Same with turning into a bytes object first:
from io import BytesIO:
file = open("zip://UTM_7N.zip", "rb")
geopandas.read_file(BytesIO(file.read()))

Traceback Here:

---

Traceback (most recent call last):
File "fiona/_shim.pyx", line 74, in fiona._shim.gdal_open_vector
File "fiona/_err.pyx", line 291, in fiona._err.exc_wrap_pointer
fiona._err.CPLE_OpenFailedError: '/vsimem/d5270a2bad8e487394a5784757b581e0/d5270a2bad8e487394a5784757b581e0' not recognized as a supported file format.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "/mnt/c/Users/victor/WKTUtils-env/lib/python3.8/site-packages/geopandas/io/file.py", line 96, in _read_file
with reader(path_or_bytes, *kwargs) as features:
File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/mnt/c/Users/victor/WKTUtils-env/lib/python3.8/site-packages/fiona/__init__.py", line 209, in fp_reader
dataset = memfile.open()
File "/mnt/c/Users/victor/WKTUtils-env/lib/python3.8/site-packages/fiona/io.py", line 73, in open
return Collection(
File "/mnt/c/Users/victor/WKTUtils-env/lib/python3.8/site-packages/fiona/collection.py", line 162, in __init__
self.session.start(self, *kwargs)
File "fiona/ogrext.pyx", line 540, in fiona.ogrext.Session.start
File "fiona/_shim.pyx", line 81, in fiona._shim.gdal_open_vector
fiona.errors.DriverError: '/vsimem/d5270a2bad8e487394a5784757b581e0/d5270a2bad8e487394a5784757b581e0' not recognized as a supported file format.

---

With the docs here, you mention you can accept any file object that has .read(). This is great for my use case, since I can't save files to disk directly. The example you have mentions StringIO, and geojson; is this failing because its a shapefile/byte object instead? Am I just calling it wrong? I tried more variations than those above, but those seemed to get me the closest, and both lead to the traceback above.

I don't know how to give a shapefile zip to go with this issue, but it has the entire set in the root, with the names the same as the .zip. Thank you in advance for any direction / help you may have to offer!

Problem description

[this should explain why the current behaviour is a problem and why the expected output is a better solution]

Shouldn't it be able to load streams, as well as files directly? If not, docs maybe need updated?

Expected Output

Expect opening the file directly, and loading it from a stream, to return the same geopanda (GeoSeries?) object.

Output of geopandas.show_versions()

Tried it again after installing these, same issue.
Failed CDLL(libgeos_c.so.1)
Failed CDLL(libgeos_c.so)

SYSTEM INFO

python : 3.8.5 (default, Jul 28 2020, 12:59:40) [GCC 9.3.0]
executable : /mnt/c/Users/victor/WKTUtils-env/bin/python3
machine : Linux-4.19.128-microsoft-standard-x86_64-with-glibc2.29

GEOS, GDAL, PROJ INFO

GEOS : None
GEOS lib : None
GDAL : 2.4.4
GDAL data dir: /mnt/c/Users/victor/WKTUtils-env/lib/python3.8/site-packages/fiona/gdal_data
PROJ : 7.2.0
PROJ data dir: /mnt/c/Users/victor/WKTUtils-env/lib/python3.8/site-packages/pyproj/proj_dir/share/proj

PYTHON DEPENDENCIES

geopandas : 0.8.1
pandas : 1.1.5
fiona : 1.8.18
numpy : 1.19.4
shapely : 1.7.1
rtree : None
pyproj : 3.0.0.post1
matplotlib : None
mapclassify: None
geopy : None
psycopg2 : None
geoalchemy2: None
pyarrow : None

[Sangarshanan]

The error occurs when the file format is wrong or when its contents are empty
You can perform file.read() only once after which it gets emptied as its already been read

# open the file
file = open("india.zip", "rb")
# This works 
geopandas.read_file(file) # Opened file has already been read
# This Fails
geopandas.read_file(file)  # Now its empty 

In the second case the output of file.read() would be an empty b' '

This was the zipfile I used https://github.com/HindustanTimesLabs/shapefiles/blob/master/india/country/india.zip

[cjshowalter]

@Sangarshanan you can do file.seek(0) to reset the read curser back to the beginning of the file. Each of those tries had data. Thank you for the file! I'll try to see if I get similar results tomorrow with it.
Edit: I didn't break apart the original scripts to best to show before. Just fixed

[Sangarshanan]

Yep would need to seek to the start every time : )

[martinfleis]

We have expanded support for file-like objects in #1535 which is not part of 0.8.1. Can you try to test with master if it resolves your issue? You can install master version of geopandas using pip directly from github.

pip install git+git://github.com/geopandas/geopandas.git

[cjshowalter]

@martinfleis Yep, that seems to have solved it perfectly. Thank you for the help!!