Cartopy: Natural Earth downloads timing out

[leecronce]

When trying to plot using Cartopy on a new installation, any Natural Earth download is timing out. In addition, if I try to download a copy of a zip file from the Natural Earth website, it times out as well. Is there an issue with the host? Here is the traceback from Python:

/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/io/__init__.py:264: DownloadWarning: Downloading: http://naciscdn.org/naturalearth/110m/physical/ne_110m_coastline.zip
warnings.warn('Downloading: {}'.format(url), DownloadWarning)
Traceback (most recent call last):
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/backends/backend_qt5agg.py", line 197, in __draw_idle_agg
FigureCanvasAgg.draw(self)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/backends/backend_agg.py", line 464, in draw
self.figure.draw(self.renderer)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
draw(artist, renderer, args, *kwargs)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/figure.py", line 1144, in draw
renderer, self, dsu, self.suppressComposite)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/image.py", line 139, in _draw_list_compositing_images
a.draw(renderer)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
draw(artist, renderer, args, *kwargs)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/mpl/geoaxes.py", line 360, in draw
inframe=inframe)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
draw(artist, renderer, args, *kwargs)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/axes/_base.py", line 2426, in draw
mimage._draw_list_compositing_images(renderer, self, dsu)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/image.py", line 139, in _draw_list_compositing_images
a.draw(renderer)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/matplotlib/artist.py", line 63, in draw_wrapper
draw(artist, renderer, args, *kwargs)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/mpl/feature_artist.py", line 134, in draw
geoms = self._feature.intersecting_geometries(extent)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/feature.py", line 116, in intersecting_geometries
return (geom for geom in self.geometries() if
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/feature.py", line 183, in geometries
name=self.name)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/io/shapereader.py", line 264, in natural_earth
return ne_downloader.path(format_dict)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/io/__init__.py", line 226, in path
result_path = self.acquire_resource(target_path, format_dict)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/io/shapereader.py", line 319, in acquire_resource
shapefile_online = self._urlopen(url)
File "/home/leec/.conda/envs/pylee/lib/python2.7/site-packages/cartopy/io/__init__.py", line 265, in _urlopen
return urlopen(url)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 467, in error
result = self._call_chain(args)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(args)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 654, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/home/leec/.conda/envs/pylee/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
URLError:

[akrherz]

See #1294 and https://downforeveryoneorjustme.com/naciscdn.org

[hafez-ahmad]

I am unable to download using cartopy

add land and coastline and 1000m bathymetry line

ax.add_feature(cfeature.LAND, facecolor='grey', zorder=1)
ax.add_feature(cfeature.COASTLINE, linewidth=0.25, zorder=1)
bathym = cfeature.NaturalEarthFeature(name='bathymetry_J_1000', scale='10m', category='physical')
bathym = cascaded_union(list(bathym.geometries()))

DownloadWarning: Downloading: http://naciscdn.org/naturalearth/10m/physical/ne_10m_bathymetry_J_1000.zip
warnings.warn('Downloading: {}'.format(url), DownloadWarning)
Traceback (most recent call last):

[dopplershift]

Please open a new issue rather than commenting on this closed one. Also, be sure to include the full error message/traceback (it's not included above).

[QuLogic]

It's not a Cartopy issue, but Natural Earth https://github.com/nvkelso/natural-earth-vector/issues/370, so opening a new issue is not necessary.

[bjlittle]

@dopplershift and @QuLogic

The http://naciscdn.org/ is proving to be not very resilient as a service recently... it's also caused travis-ci to grind to a halt on scitools/iris... makes me think standing-up a mirror service that scrapes http://naciscdn.org/ for content when it is available, and hitting that instead (automatically on an naciscdn.org outage) might be something to consider going forwards.

What do you guys think?

Otherwise, I really do think that this is going to happen again, and again... which is a royal pain for the community.

[bjlittle]

The other alternative is to make the cartopy data available as a conda package, akin to https://github.com/conda-forge/iris-sample-data-feedstock or explore/extend https://github.com/conda-forge/cartopy_offlinedata-feedstock.

Just floating some ideas... :thinking:

[greglucas]

Looks like another issue is looking into it with more recent updates: https://github.com/nvkelso/natural-earth-vector/issues/372

Another thought is to write a thin wrapper that downloads all of the contents of the zip files from the main GitHub repo (they aren't zipped up there which would have enabled the mirror approach easily).
https://github.com/nvkelso/natural-earth-vector

Creating a conda package with the basic data in addition to a more robust mirror would also potentially be useful. See #1325 for interest in getting the data on a cluster without internet access.

[dopplershift]

@greglucas That jogged a memory--that package already exists: cartopy_offlinedata.

>conda search cartopy_offlinedata    
Loading channels: done
# Name                       Version           Build  Channel             
cartopy_offlinedata            0.2.1            py_0  conda-forge         
cartopy_offlinedata            0.2.1            py_1  conda-forge         
cartopy_offlinedata            0.2.1    pyh9f0ad1d_2  conda-forge         
cartopy_offlinedata            0.2.2    pyh9f0ad1d_0  conda-forge         
cartopy_offlinedata            0.2.3    pyh9f0ad1d_0  conda-forge         

Edit: I see @bjlittle said that now.