Rcpp: Issue with RcppModules on OS X and gcc/g++

[dselivanov]

I have strange issues with installing packages which contain Rcpp Modules code on OS X with gcc-6/g++-6. I tried on at least 2 packages - text2vec, rcppannoy.

sessionInfo()

R version 3.3.1 (2016-06-21)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: OS X 10.12 (Sierra)
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base

Compilation actually goes successfully. But "testing if installed package can be loaded" phase gives following (same process with clang/clang++ - no problems!)

devtools::install_github("eddelbuettel/rcppannoy")

.........
* R
** data
* moving datasets to lazyload DB
** inst
** preparing package for lazy loading
** help
** installing help indices
* building package indices
** installing vignettes
** testing if installed package can be loaded
sh: line 1: 71168 Segmentation fault: 11 '/Library/Frameworks/R.framework/Resources/bin/R' --no-save --slave 2>&1 < '/var/folders/82/52xsqd6d3m9031wwnr5z4yyh0000gp/T//RtmpR3pYRG/file115b0581b2849'
** caught segfault *
address 0x20, cause 'memory not mapped'
Traceback:
1: .Call(Module__classes_info, xp)
2: Module(module, mustStart = TRUE, where = env)
3: doTryCatch(return(expr), name, parentenv, handler)
4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
5: tryCatchList(expr, classes, parentenv, handlers)
6: tryCatch(Module(module, mustStart = TRUE, where = env), error = function(e) e)
7: loadModule(module = "VocabCorpus", what = TRUE, env = ns, loadNow = TRUE)
8: (function (ns) loadModule(module = "VocabCorpus", what = TRUE, env = ns, loadNow = TRUE))()
9: doTryCatch(return(expr), name, parentenv, handler)
10: tryCatchOne(expr, names, parentenv, handlers[[1L]])
11: tryCatchList(expr, classes, parentenv, handlers)
12: tryCatch((function (ns) loadModule(module = "VocabCorpus", what = TRUE, env = ns, loadNow = TRUE))(), error = function(e) e)
13: eval(expr, envir, enclos)
14: eval(substitute(tryCatch(FUN(WHERE), error = function(e) e), list(FUN = f, WHERE = where)), where)
15: .doLoadActions(where, attach)
16: methods::cacheMetaData(ns, TRUE, ns)
17: loadNamespace(package, lib.loc)
18: doTryCatch(return(expr), name, parentenv, handler)
19: tryCatchOne(expr, names, parentenv, handlers[[1L]])
20: tryCatchList(expr, classes, parentenv, handlers)
21: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call)[1L] prefix <- paste("Error in", dcall, ": ") LONG <- 75L msg <- conditionMessage(e) sm <- strsplit(msg, "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && identical(getOption("show.error.messages"), TRUE)) { cat(msg, file = stderr()) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))})
22: try({ attr(package, "LibPath") <- which.lib.loc ns <- loadNamespace(package, lib.loc) env <- attachNamespace(ns, pos = pos, deps)})
23: library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)
24: withCallingHandlers(expr, packageStartupMessage = function(c) invokeRestart("muffleMessage"))
25: suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE))
26: doTryCatch(return(expr), name, parentenv, handler)
27: tryCatchOne(expr, names, parentenv, handlers[[1L]])
28: tryCatchList(expr, classes, parentenv, handlers)
29: tryCatch(expr, error = function(e) { call <- conditionCall(e) if (!is.null(call)) { if (identical(call[[1L]], quote(doTryCatch))) call <- sys.call(-4L) dcall <- deparse(call)[1L] prefix <- paste("Error in", dcall, ": ") LONG <- 75L msg <- conditionMessage(e) sm <- strsplit(msg, "\n")[[1L]] w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w") if (is.na(w)) w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L], type = "b") if (w > LONG) prefix <- paste0(prefix, "\n ") } else prefix <- "Error : " msg <- paste0(prefix, conditionMessage(e), "\n") .Internal(seterrmessage(msg[1L])) if (!silent && identical(getOption("show.error.messages"), TRUE)) { cat(msg, file = stderr()) .Internal(printDeferredWarnings()) } invisible(structure(msg, class = "try-error", condition = e))})
30: try(suppressPackageStartupMessages(library(pkg_name, lib.loc = lib, character.only = TRUE, logical.return = TRUE)))
31: tools:::.test_load_package("text2vec", "/Users/dymitriyselivanov/Library/R/3.3/library")
An irrecoverable exception occurred. R is aborting now ...
ERROR: loading failed

• removing ‘/Users/dymitriyselivanov/Library/R/3.3/library/text2vec’
• restoring previous ‘/Users/dymitriyselivanov/Library/R/3.3/library/text2vec’

On other hand I tried the same on ubuntu with gcc/g++-6 (6.2) - no problems.

[eddelbuettel]

Oh dear. I could try on Debian unstable as well -- we now build all packages in the distribution with that compiler version. (And I believe Prof Ripley already runs tests with the unreleased gcc7.)

So probably an OS X issue. Not sure how we can go about this. Any idea, @kevinushey ?

[dselivanov]

Some time ago I tried with gcc-5 - same errors. Seems problem is with os x. I was happy with clang, but yesterday discovered that gcc optimize code much better - 1.5-2x faster than clang...
Also discovered that R doesn't properly forward compiler flags for C++11 code on os x (due to bad system Makevars.site file). Will write to r-devel today.
Edit: I was wrong - before submitting r-devel double checked. Had a typo in one letter of CXX1XFLAGS :-) Battle Tested Makevars.site - 👍

[kevinushey]

The problem here is likely that you are mixing compilers / toolchains. I get the same error as you if I try to mix a clang-compiled Rcpp with a gcc-compiled RcppAnnoy. As much as clang and gcc try to be interoperable, it doesn't always work -- so if you mix toolchains, and things break, you get to keep the pieces.

Installing Rcpp compiled with gcc-6 first, and then installing RcppAnnoy, works successfully.

[eddelbuettel]

That seems more like it. @dselivanov: ok to close?

[dselivanov]

Let me check. I will return shortly.

6 окт. 2016 г. 9:10 ПП пользователь "Dirk Eddelbuettel" <
[email protected]> написал:

That seems more like it. @dselivanov https://github.com/dselivanov: ok
to close?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/RcppCore/Rcpp/issues/555#issuecomment-252027479, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AE4u3YnjLtatyegF1RXc7Vupak2d2qTMks5qxStxgaJpZM4KPtwZ
.

[eddelbuettel]

No rush. We can close tomorrow, or next week. I just don't want it to linger "indefinitely".

[dselivanov]

Thanks @kevinushey for solution. Installed packages chain with g++-6 and all works fine!
With g++-6 code run 30-50% faster than with apple clang++.
Thanks @eddelbuettel for support.

[eddelbuettel]

Wow. 30-50% matters. Nice. Do tweet or blog about it :)

[dselivanov]

More interesting thing is that optimization -O3 (instead of default -O2 ) with clang compiler made things much slower! In my case almost 2-2.5x... Weird! See this topic if interested.

[kevinushey]

If you have time, it'd be worth benchmarking against clang-3.8 (as provided by Homebrew with brew install llvm38), or even clang-4.0 (if you're feeling brave enough to build clang from sources).

[dselivanov]

@kevinushey @eddelbuettel sorry for bothering you again.
Is it possible to catch such situations (*** caught segfault ***, address 0x20, cause 'memory not mapped') from inside pkg during installation and output some advise how to solve the issue? I don't think this is possible, but may be there is some trick...

[eddelbuettel]

Not that I know of.