Geopandas: Error using to_crs() -- no such file or directory - ONLY on windows

[lwasser]

Hi Colleagues.
Some of my students are running into errors when they reproject using this code:

object.to_crs(another_object.crs)

RuntimeError                              Traceback (most recent call last)
<ipython-input-5-eb6d1da3880a> in <module>()
----> 1 sjer_aoi_wgs84  = sjer_aoi.to_crs(sjer_roads.crs)

~\Anaconda3\envs\earth-analytics-python\lib\site-packages\geopandas\geodataframe.py in to_crs(self, crs, epsg, inplace)
    441         else:
    442             df = self.copy()
--> 443         geom = df.geometry.to_crs(crs=crs, epsg=epsg)
    444         df.geometry = geom
    445         df.crs = geom.crs

~\Anaconda3\envs\earth-analytics-python\lib\site-packages\geopandas\geoseries.py in to_crs(self, crs, epsg)
    302             except TypeError:
    303                 raise TypeError('Must set either crs or epsg for output.')
--> 304         proj_in = pyproj.Proj(self.crs, preserve_units=True)
    305         proj_out = pyproj.Proj(crs, preserve_units=True)
    306         project = partial(pyproj.transform, proj_in, proj_out)

~\Anaconda3\envs\earth-analytics-python\lib\site-packages\pyproj\__init__.py in __new__(self, projparams, preserve_units, **kwargs)
    356         # on case-insensitive filesystems).
    357         projstring = projstring.replace('EPSG','epsg')
--> 358         return _proj.Proj.__new__(self, projstring)
    359 
    360     def __call__(self, *args, **kw):

_proj.pyx in _proj.Proj.__cinit__()

RuntimeError: b'No such file or directory'

i have not been able to recreate it on my mac but can on windows 10.
I found this issue which looks related: https://github.com/jswhit/pyproj/issues/101

and this SE post

https://gis.stackexchange.com/questions/295914/error-in-coordinate-transformation-with-geopandas

to which @jorisvandenbossche suggested removing proj4 and pyproj. i tried that but it did not fix the issue. also when you remove proj4 you have to remove gdal and rasterio and cartopy... so all of a sudden my environment is gone. I also did a full fresh reinstall of the environment.

Any ideas how I can resolve this issue? And why is it so windows specific!? note - i'm on a pretty fresh windows install and do not have arcgis or anything else here. just git, python / conda and a few other basic tools for text editins and such. im using python 3.x.

Many thanks for any suggestions.

[jorisvandenbossche]

And the full fresh reinstall of the environment did also not work?

Do you have an overview of how the environment is created?

Can you check the value of pyproj.pyproj_datadir ?

cc @ocefpaf

[lwasser]

Hi @jorisvandenbossche thank you so much for the speedy reply.
When i run pyproj.pyproj_datadir it returns
' ' so nothing. i presume that is not the expected return?

My environment that all students use is here:

https://github.com/earthlab/earth-analytics-python-env/blob/master/environment.yml

it has actually worked well for about a year on all OS's. BUT i can't say when this particular crs issue popped up.

For my test, i actually have a VM running windows. i installed anaconda and the environment last night. it was a completely clean install of all things python. And with that install i was finally able to recreate the students issues.

If you have any other suggestions of things i should try / print out / etc please say the word.

[jdmcbr]

This actually looks related, though distinct, to what @jwilson8767's PR in https://github.com/geopandas/geopandas/pull/678 is seeing on our AppVeyor build. Fresh install on windows, and the pyproj_datadir isn't being found (or at least the epsg sub-directory), though it is not returning an empty string, and all the reprojection tests actually are passing.

[lwasser]

I'm happy to help if i can. If it's useful for me to test things on my windows VM please just let me know what to try. @jdmcbr @jorisvandenbossche

[jwilson8767]

So I poked back through conda-forge builds of pyproj for win64 and found that builds since this commit have lacked the data directory entirely (where they were present in builds prior to that commit). It appears that this commit 3 days ago may have fixed this issue, but I'm not sure as there are no conda-forge builds since then.

I don't know conda-forge, is that something we can just conda smithy rerender to get a new build out for this version of pyproj?

[lwasser]

hey colleagues. I just pinged @ocefpaf on that commit above to see if perhaps a fix was in the works or in place already on conda-forge. If this isn't resolvable in the near future, would specifying a different version of pyproj potentially resolve this issue for my students running windows? thank you so much!

[jwilson8767]

@lwasser You should be able to lock the pyproj version to 1.9.4 or specifically using this build for your students case.

[jorisvandenbossche]

One thing that should work to solve it temporarily is to set the PROJ_LIB environment variable before importing pyproj.

[lwasser]

ok great! i'll try it with v 1.9.4 today @jwilson8767 thank you so much! @jorisvandenbossche i'm curious. how / where would i set the PROJ_LIB envt variable? would this entail somehow downloading that missing / empty library and then directing python to the path where it is located on the users machine? i'm so grateful for your help with this.

Many thanks to you both!

[jorisvandenbossche]

One way is to do that from python itself:

import os
os.environ["PROJ_LIB"] = "/path/to/lib/data"

or otherwise you can set it on the command line before starting the python console / notebook.

The data should be there, it is only that pyproj doesn't find them (as far as I understood, I might be wrong)

[jwilson8767]

@jorisvandenbossche nope, data is entirely missing from the build in this case.

[jorisvandenbossche]

In the pyproj one, but it should still be included via the proj package.

[jorisvandenbossche]

On linux this is located in .../Anaconda3/envs/env_name/share/proj, not sure by heart what the windows equivalent is exactly

[gboeing]

On my machines I got it working via:

import os
os.environ["PROJ_LIB"] = "C:\Anaconda\envs\env_name\Library\share" #windows
os.environ["PROJ_LIB"] = ".../Anaconda/envs/env_name/Library/share" #linux

[lwasser]

i will give this a go as well. thanks @gboeing ! i was able to "downgrade" pyproj to 1.9.4 and that worked BUT i didn't like doing it as it also modified dozens of other packages in my environment which always makes me worry! it did work however. id prefer your method if it works for me!!

[jtamerius]

On my machines I got it working via:

import os
os.environ["PROJ_LIB"] = "C:\Anaconda\envs\env_name\Library\share" #windows
os.environ["PROJ_LIB"] = ".../Anaconda/envs/env_name/Library/share" #linux

This worked for me. Thanks @gboeing !

[lwasser]

BINGO!! i updated my envt and ran the above and it worked like a charm!! thank you again @gboeing @jorisvandenbossche i much prefer this solution as it allows me to keep my envt in tact and up to date!!

[MonaHemmati]

Hi all,

I tried everything mentioned in forums regarding this question. I couldn't solve the problem with the suggested solution. What I did and solved my problem was :1) close all environment of Python you are using. 2)Updating Anaconda Navigator 3) Launch your interest programming Env. from Anaconda navigator (Spyder, Jupyter Notebook, and etc)directly after it gets updated

I followed the mentioned steps and I didn't get any error. :)

[danieljbradley]

Hi all,

I tried everything mentioned in forums regarding this question. I couldn't solve the problem with the suggested solution. What I did and solved my problem was :1) close all environment of Python you are using. 2)Updating Anaconda Navigator 3) Launch your interest programming Env. from Anaconda navigator (Spyder, Jupyter Notebook, and etc)directly after it gets updated

I followed the mentioned steps and I didn't get any error. :)

@MonaHemmati This worked for me too. Thanks for sharing! #3 is was necessary for me. I forgot that one and launched jupyter notebook from command line and I kept getting the error. Once I launched from AN, to_crs worked. For #2, Anaconda Navigator did not seem to be updating. On Windows, I launched Powershell (right click "Run as Administrator") and ran these commands:

conda deactivate
conda update anaconda-navigator

and it updated pretty quickly.

[lwasser]

oops -- just read the above. you are on windows. my students are still running into this issue and i wonder if i can ask them to try this as a fix! i will make a note of it if it works, here...