Sf: Error reading .kmz file from vignette

[pssguy]

library(sf)
u_kmz <- "http://coagisweb.cabq.gov/datadownload/BikePaths.kmz"download.file(u_kmz, "BikePaths.kmz") 

biketrails_kmz <- st_read("BikePaths.kmz")
Error in CPL_read_ogr(dsn, layer, as.character(options), quiet, iGeomField -  : 
Open failed.

 sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252    LC_MONETARY=English_Canada.1252 LC_NUMERIC=C                   
[5] LC_TIME=English_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] sf_0.2-7

loaded via a namespace (and not attached):
[1] DBI_0.5-1   tools_3.3.1 Rcpp_0.12.8 grid_3.3.1 

I can confirm the BikePaths.kmz is in folder

TIA

[etiennebr]

It works for me on version 0.2-8 from development. It plots nicely ! If you can, install it:

library(devtools)
install_github("edzer/sfr")

Otherwise, if you have gdal installed, you could try system('ogr2ogr -f "ESRI Shapefile" BikePaths.shp BikePaths.kmz') and read st_read("BikePaths.shp").

[pssguy]

@etiennebr Thanks . I'm on 0.2-7 from CRAN - but then so is the vignette!

Tried installing the dev version

Error : .onAttach failed in attachNamespace() for 'sf', details:
  call: .Call("sf_CPL_proj_version", PACKAGE = "sf", b)
  error: "sf_CPL_proj_version" not available for .Call() for package "sf"

Also get the same, 'Error in CPL_read_ogr' with a different .shp file I wanted to load

[edzer]

I get no problems reading this file with 0.2-7. here it says that "KML reading is only available if GDAL/OGR is built with the Expat XML Parser, otherwise only KML writing will be supported." I see

> st_drivers()[row.names(st_drivers()) == "KML",]
    name                     long_name write  copy is_raster is_vector
KML  KML Keyhole Markup Language (KML)  TRUE FALSE     FALSE      TRUE

what do you see there?

[pssguy]

I get the same result. as you. The vignettes say the assumption is all drivers can read. I'm pretty naive in this area and not sure where the CPL_read_ogr function comes from

BTW I moved and renamed my shp (and other associated files) and that now does not appear to be a problem

[edzer]

Are you sure the .kmz file actually downloaded? The code segment in your issue does not execute, as it is. As a check, could you tell me how large the downloaded file is?

[pssguy]

Hope this helps

[dieza]

Same happens to me on mac.
biketrails_kmz <- st_read("BikePaths.kmz")
Cannot open data source /Users/agus/BikePaths.kmz
Error in CPL_read_ogr(dsn, layer, as.character(options), quiet, iGeomField - :
Open failed.

But

ogr2ogr -f "ESRI Shapefile" BikePaths.shp BikePaths.kmz

and

st_read("BikePaths.shp")

works

Reading layer BikePaths' from data source/Users/agus/BikePaths.shp' using driver `ESRI Shapefile'
Simple feature collection with 3324 features and 12 fields
geometry type: GEOMETRY
dimension: XYZ
bbox: xmin: -106.7982 ymin: 34.78517 xmax: -106.2212 ymax: 35.37142
epsg (SRID): 4326
proj4string: +proj=longlat +datum=WGS84 +no_defs

sessionInfo

R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.1

locale:
[1] ca_ES.UTF-8/ca_ES.UTF-8/ca_ES.UTF-8/C/ca_ES.UTF-8/ca_ES.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] sp_1.2-3 RPostgreSQL_0.4-1 DBI_0.5-1 sf_0.2-7

loaded via a namespace (and not attached):
[1] rgdal_1.1-10 tools_3.3.2 Rcpp_0.12.8 grid_3.3.2 lattice_0.20-34

[edzer]

What does

library(rgdal)
x = readOGR("BikePaths.kmz", "BikePaths")

give you?

[pssguy]

@edzer
I get

```

x = readOGR("BikePaths.kmz", "BikePaths")
Error in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv, :
Cannot open data source
``

[edzer]

Looking again, on my machine reading reports

Reading layer `BikePaths' from data source `/tmp/BikePaths.kmz' using driver `LIBKML'
...

so not driver KML but driver LIBKML; what do you see when you ask

> st_drivers()[row.names(st_drivers()) == "LIBKML",]

?

[pssguy]

```

st_drivers()[row.names(st_drivers()) == "LIBKML",]
[1] name long_name write copy is_raster is_vector
<0 rows> (or 0-length row.names)

[edzer]

That's what I suspected. What you may want to try is:

1. rename BikePaths.kmz to BikePaths.kml.zip
2. unzip this file
3. read the unzipped file with st_read("doc.kml")

I think that reading .kmz (i.e.. zipped .kml's) files requires libkml (which your gdal lib doesn' t have), and that the kml driver (not the libkml driver) only reads .kml.

[pssguy]

The unzipping does not work after renaming

"BikePaths.kml.zip is invalid"

[edzer]

From here on, the problems are not related to the sf package, so I suggest to close the issue.

[pssguy]

OK so are you saying kml files cannot be loaded without gdal which is not available for Windows?

[edzer]

Windows (and Mac) binary CRAN packages don't link to external user software (your link to gdal), but come with statically linked gdal/geos/proj, which explains the large size of the .dll in the package lib. This prepackaged gdal does not have libkml (as documented), and hence does not read .kmz files. You can either

1. rename the .kmz, unzip to .kml, and read with the KML (not LIBKML) driver, as explained above or
2. install gdal yourself and install sf _from source_, linking to your gdal installation at install time.

[pssguy]

Thanks for your time
As mentioned above, 1 did not work for me re unzipping
I'll report on 2 if and when

[etiennebr]

I think the vignette should be fixed, or there should be a note, because this situation will likely happen a hundred times given Windows and CRAN popularity :)

[dieza]

Sorry for catching up a little later. BikePaths.kmz can be read on my mac but unzipping allows to read the result kml.

x<-readOGR("BikePaths.kmz","Bikepaths")

Error in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv, :
Cannot open data source
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.2

locale:
[1] ca_ES.UTF-8/ca_ES.UTF-8/ca_ES.UTF-8/C/ca_ES.UTF-8/ca_ES.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] rgdal_1.2-5 sp_1.2-4 sf_0.2-8 readODS_1.6.4 wordcloud_2.5 RColorBrewer_1.1-2 SnowballC_0.5.1
[8] tm_0.6-2 NLP_0.1-9

loaded via a namespace (and not attached):
[1] Rcpp_0.12.8 lattice_0.20-34 assertthat_0.1 slam_0.1-40 grid_3.3.2 cellranger_1.1.0 DBI_0.5-1
[8] units_0.4-1 xml2_1.1.0 tools_3.3.2 udunits2_0.13 readr_1.0.0 parallel_3.3.2 tibble_1.2

El 6Jan, 2017, a las 2:23 AM, Etienne B. Racine [e-mail] escribi�:

I think the vignette should be fixed, or there should be a note, because this situation will likely happen a hundred times given Windows and CRAN popularity :)

�
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[edzer]

@Robinlovelace could you pls take a look and suggest an update to the vignette?

[Robinlovelace]

Sure - my bad, wanted to add a real world data set so it's not purely theoretical.

To summarize the situation: will use a .kml file for benchmarking instead - plan to find a kml file from the internet and use that instead.

A wider question: I think there is a need to set-up a directory of example spatial data files. Discussed this with @Nowosad. So could kill 2 birds with one stone by creating such a data repo, perhaps eventually to be submitted to CRAN for people wanting explore spatial data types and datasets and do benchmarking. That way we're not relying on other peoples' servers (which could go down) and could have a spatial equivalent of the datasets package http://stat.ethz.ch/R-manual/R-devel/library/datasets/html/datasets-package.html and providing a range of data formats, like these examples from the rio package: https://github.com/leeper/rio/tree/master/inst/examples

Sound good? Happy to do a quick fix to close this and then look to compile a collection of small yet diverse spatial datasets for benchmarking and other purposes. If something like this exists already please let me know so I don't reinvent any wheels.

[edzer]

Good idea. This would be a separate R package, right?

[rsbivand]

I've been approached by Debian people before about something like this, because of uncertainties about licencing. Having a separate package with example data sets would follow this up. The attempt that I began is at https://r-forge.r-project.org/scm/viewvc.php/pkg/spDatasets/?root=rspatial with datasets from spdep and maptools (maybe more) in RData file format and sp classes.

[Robinlovelace]

Fantastic, many thanks for sharing that existing package Roger. I could add to it rather than create another one - sound good? Or would it be more advantageous to create another package focussed on, but not limited to, file formats, e.g. called spDataForms.

[rsbivand]

You are very welcome to build on this if it is at all useful. The idea was to collect data sets and pre-package them as sp objects to avoid having to read them in each time, but I never got as far as releasing it and shifting spdep and maptools to suit.

These are quotations from an email exchange in 2014:

The purpose of all the packages I maintain is teaching at graduate
levels, and all the examples (so the data sets) are included in the
packages to permit users to reproduce text book results. This
software is not an application, it is intended to permit learning,
first by doing what text books say, then learning from own data.
This is my ontology. Consequently, the licensing/copyright of the
code and its documentation is orthogonal to that of the data sets.

Note that spdep has far more data than maptools, but this has not
been questioned. If the data license issues in spdep are not a
problem, why were they a problem in maptools? Is GPL a relevant
license for data sets (rhetorical question)? The examples on the
help pages all need access to data sets - the data sets chosen by me
are those that are most relevant for relating the methods to text
books in which they are discussed.

One (bad/ugly) resolution is to create a separate data set package.
However, this is a general problem because most CRAN packages face
the same problem.

By the way, I maintain maptools, and cannot recall being contacted
in 2009. Nicholas contributed code in 2000/2001, but now has nothing
to do with maptools.

The issue was that Debian required all files to be free to change, whereas I suggested that changing the content of data files makes no great sense. I ended up in my draft package with a CC licence.

[Robinlovelace]

Sounds great - will aim to build on that one. Only issue, I seem incapable of cloning the svn repo, even with this: https://gist.github.com/philchalmers/7281255#file-rforge2git-r-L9 - any chance you could dump it on GitHub or something similar - may just be my account's not set-up yet on R-Forge.

[pavelsjo]

That's what I suspected. What you may want to try is:

1. rename `BikePaths.kmz` to `BikePaths.kml.zip`

2. unzip this file

3. read the unzipped file with `st_read("doc.kml")`

I think that reading .kmz (i.e.. zipped .kml's) files requires libkml (which your gdal lib doesn' t have), and that the kml driver (not the libkml driver) only reads .kml.

Thanks a lot!