Sf: set up a docker file for testing with lowest version of system requirements

[edzer]

i.e. gdal 2.0.0, proj 4.8.0 and geos 3.4.0 -- we now see errors on fedora CRAN checks probably caused by it having gdal 2.1.0 rather than 2.2.x.

[edzer]

I'm getting stuck at getting rgdal to install with proj 4.8.0; this breaks at

checking projects.h presence and usability... no
configure: error: projects.h not found in standard or given locations.
ERROR: configuration failed for package 'rgdal'
* removing '/usr/local/lib/R/site-library/rgdal'

and it seems that proj 4.8.0 does indeed not install this header file. Both CRAN and r-forge versions of rgdal are affected. @rsbivand do you want to increase the sys req version of proj, or backport this?

[edzer]

... current work-around:
https://github.com/r-spatial/sf/blob/cf1f55934ba9661dae65cd5375fbaa62ebb6f8b3/inst/docker/lowest/Dockerfile#L52-L53

[rsbivand]

Am working on checking rgdal against its minimum requirements. R-forge rgdal does check for presence and usability of projects.h (waiting to CRAN processing).

[edzer]

Yes, but it gives an error if it is not found (line 256, configure.ac).

[rsbivand]

Getting older GDAL to compile is also very hard, as ancient C++ code was abundant, and without containers with legacy OS to hand effectively impossible, even using -std=.

[edzer]

Right; I had to move to 2.0.1 instead of 2.0.0, and used an ubuntu 16.04 docker image. For older GDAL versions I think I would start off with an 14.04 image, or 12.04, which are all ready available.

[rsbivand]

The error is necessary IMO, and what was what was requested by the questioner (on list?). Otherwise it breaks during compilation; before Barry's test, R would hard crash when project() was used with no inverse provided. From 4.9.3, proj itself provided the protection. I'll probably move to strech gdal 2.1.2 proj 4.9.3 and fedora 25 gdal 2.1.0 proj 4.9.2 (still unprotected). Otherwise I'd have to provide gcc 4* compilers, and don't feel motivated to do that. I could provide a flag to give control of the project.h availability test to the user, issuing a warning if they need to make the headers available. I can't build GDAL 1.11.4 or earlier even with -std=c++0x.

[rsbivand]

The issues with rgdal check also include EPSG codes for older PROJ, so lots of conditioning would be needed to make a gdal 1.6.3 proj 4.4.9 system complete check without error for current rgdal. I'll update DESCRIPTION to tell users with outdated systems to use an rgdal version from the same time period.

[edzer]

Sounds good!

[edzer]

While at it, and checking sf against proj 5.0.1 and gdal 2.3.0beta1, rgdal breaks because a c++11 compiler is required:

checking gdal: linking with --libs and --dep-libs... no
In file included from /usr/local/include/gdal.h:45:0,
                 from gdal_test.cc:1:
/usr/local/include/cpl_port.h:187:6: error: #error Must have C++11 or newer.
 #    error Must have C++11 or newer.
      ^

[ezwelty]

@edzer Not necessarily the appropriate place to report this, but I got the exact same error trying to install sf on my system:

• Mac OS X 10.11.6
• R 3.3.3
• gdal 2.3.0
• proj 5.1.0
• rgdal: 1.3-3

install.packages("sf")

In file included from gdal_test.cpp:1:
In file included from /usr/local/Cellar/gdal/2.3.0/include/gdal.h:45:
/usr/local/Cellar/gdal/2.3.0/include/cpl_port.h:187:6: error: Must have C++11 or newer.
#    error Must have C++11 or newer.
     ^

After putting this in ~/.R/Makevars

CC=clang
CXX=clang++
PKG_CXXFLAGS= -stdlib=libc++ -std=c++11

and running

devtools::install_github("r-spatial/sf")

it worked.

I won't be able to investigate if it was Makevars or installing from dev that made the difference, or whether something else broke in its stead, as I have a dissertation to defend in a couple weeks, but found this thread searching for the original error and thought I would share in case it's useful.

[kevinrue]

Different story, same solution, this worked for me:

devtools::install_github("cran/rgdal")

when install.packages("rgdal") failed with the error message error Must have C++11 or newer.

Just thought I'd share the solution with the web, as the relevant repo is read-only and doesn't accept issues (https://github.com/cran/rgdal).

[rsbivand]

@kevinrue Please always give full platform specifications and rgdal version details. As you say (too low down), this is in no way the same problem. If you tried to install CRAN rgdal from source on a linux platform yesterday, you almost certainly got 1.3-4, but the totally unofficial github mirror remains at 1.3.3 in part, and spattered with older copies elsewhere. Never, ever, use the github mirror for anything if you are part of the community, nothing from there is relevant or supported in any way. Consult the development repo on R-Forge. Do not spread unverifiable rumours; the need for CXX11 in rgdal only applies if your GDAL is >=2.3 (which requires CXX11 in sf and rgdal). This typically only happens on really old CentOS or RHEL without a CXX11-capable default compiler and GDAL >= 2.3.

If you can provide full platform details (OS, c++ compiler version, GDAL and PROJ versions, R version, version of rgdal you downloaded from CRAN), and after comparing with https://cran.r-project.org/web/checks/check_results_rgdal.html still maintain that there is an issue, run ./configure inside the unpacked source tree and provide the config.log output.

[kevinrue]

@rsbivand
Dear Roger,
Apologies for the lack of details in my previous post.

My system:

• Mac OS X 10.13.6
• R 3.5.1
• gdal: 2.3.1
• proj 5.1.0

I was using the CRAN package BiocManager and its function valid() was telling me about rgdal being out of date (Unfortunately I didn't write down the original rgdal version ).

As a consequence, I tried a few times the suggested command:

 BiocManager::install("rgdal", update = TRUE, ask = FALSE)

which failed with the error described by ezwelty.

That is when somehow the devtools command passed successfully.
Strangely enough today I cannot reproduce the issue anymore. The BiocManager command above works just fine.

My excuses again for the confusion. Best, wishes.

[rsbivand]

My best guess is that BiocManager() with the arguments given tried a source install of 1.3-4 because the 1.3-4 CRAN OSX binary had not yet been built, but that today the CRAN OSX binary had been built. For OSX the R-sig-mac list is the place to look; for rgdal R-sig-geo. Do you have your own PROJ and/or GDAL that you have installed yourself? Since the install failure cannot be reproduced, it is hard to diagnose.

[kevinrue]

I have installed gdal using brew install gdal, which also seems to have taken care of installing proj (the timestamps of the executables are all from yesterday).

That was my first time (knowingly) installing rgdal and thus gdal, proj, etc because of a new dependency in an R package that I use. So I didn't really know where the issue was originating from.

As you mention, perhaps just a poor timing on my side. Not the first time this would have happened to me. Thanks for your insights though, I'll make sure to include more details in future issues.

[rsbivand]

OK, I understand. We can't do too much about packages choosing to depend on or import rgdal, though if rgdal isn't an important part of the workflow, that package's author might rather have chosen to suggest, so not provoking a difficult installation. On OSX we prefer to see the CRAN binary used, which should be fairly robust. Maybe see this thread: https://stat.ethz.ch/pipermail/r-sig-mac/2018-August/012762.html, pointing up an omission in the current CRAN OSX rgdal binary, but one which may also affect you (depending on whether brew installed the PROJ datum metadata files). They are used when heights above or below the ellipsoid differ between coordinate reference systems.

[kevinrue]

Thank you.
Following the post you've linked, I've run

> projNAD()
[1] FALSE

which was subsequently resolved by the rest of your post:

pkg <- "rgdal"
exdir <- file.path(system.file(package=pkg), "proj")
list.files(exdir)
tf <- tempfile()
fl <- "http://download.osgeo.org/proj/proj-datumgrid-1.7.zip"
download.file(fl, tf)
unzip(tf, exdir=exdir)
list.files(exdir)
 [1] "alaska"             "BETA2007.gsb"       "CH"                 "conus"             
 [5] "egm96_15.gtx"       "epsg"               "esri"               "esri.extra"        
 [9] "FL"                 "GL27"               "hawaii"             "IGNF"              
[13] "MD"                 "nad.lst"            "nad27"              "nad83"             
[17] "ntf_r93.gsb"        "ntv1_can.dat"       "null"               "nzgd2kgrid0005.gsb"
[21] "other.extra"        "proj_def.dat"       "prvi"               "README.DATUMGRID"  
[25] "stgeorge"           "stlrnc"             "stpaul"             "TN"                
[29] "WI"                 "WO"                 "world"             
> projNAD()
[1] TRUE

Best wishes,
Kevin

[rsbivand]

@ezwelty made a suggestion above that is probably incorrect (at least for released and development R). The doubtful suggestion (above) was:

CC=clang
CXX=clang++
PKG_CXXFLAGS= -stdlib=libc++ -std=c++11

but R CMD config CXX on clang platforms returns 'clang++ -std=gnu++11'. This thread on R-sig-geo is the documentation (read all three postings).