Data.table: R 3.3.3 data.table v1.10.5 install fails with: Library not loaded: @rpath/libomp.dylib, CRAN ok but single core

Created on 12 Mar 2017 · 10Comments · Source: Rdatatable/data.table

After last Mac Sierra available updates... some issue with OpenMP.

History and log:

fresh R reinstall after:
sudo rm -rf /Library/Frameworks/R.framework /Applications/R.app \
/usr/bin/R /usr/bin/Rscript
"OpenMP enabled compiler for Mac" steps

robs-MBP:.R rob$ xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates
brew upgrade && brew update && brew install llvm - done 
Already up-to-date.
~/.R/Makevars - done 
robs-MBP:.R rob$ cat Makevars
CC=/usr/local/opt/llvm/bin/clang -fopenmp
CXX=/usr/local/opt/llvm/bin/clang++
LDFLAGS=-L/usr/local/opt/gettext/lib -L/usr/local/opt/llvm/lib
CPPFLAGS=-I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include

installation:

> remove.packages("data.table")
Removing package from ‘/Users/rob/Library/R/3.3/library’
(as ‘lib’ is unspecified)
Error in find.package(pkgs, lib) :
  there is no package called ‘data.table’
> install.packages("data.table", type = "source",
+     repos = "http://Rdatatable.github.io/data.table")
Installing package into ‘/Users/rob/Library/R/3.3/library’
(as ‘lib’ is unspecified)
trying URL 'http://Rdatatable.github.io/data.table/src/contrib/data.table_1.10.5.tar.gz'
Content type 'application/octet-stream' length 2841831 bytes (2.7 MB)
==================================================
downloaded 2.7 MB

* installing *source* package ‘data.table’ ...
** libs
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c assign.c -o assign.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c between.c -o between.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c bmerge.c -o bmerge.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c chmatch.c -o chmatch.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c dogroups.c -o dogroups.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fastmean.c -o fastmean.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fcast.c -o fcast.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fmelt.c -o fmelt.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c forder.c -o forder.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c frank.c -o frank.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fread.c -o fread.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fsort.c -o fsort.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c fwrite.c -o fwrite.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c gsumm.c -o gsumm.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c ijoin.c -o ijoin.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c init.c -o init.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c inrange.c -o inrange.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c openmp-utils.c -o openmp-utils.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c quickselect.c -o quickselect.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c rbindlist.c -o rbindlist.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c reorder.c -o reorder.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c shift.c -o shift.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c subset.c -o subset.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c transpose.c -o transpose.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c uniqlist.c -o uniqlist.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c vecseq.c -o vecseq.o
/usr/local/opt/llvm/bin/clang -fopenmp -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include    -fPIC  -Wall -mtune=core2 -g -O2  -c wrappers.c -o wrappers.o
/usr/local/opt/llvm/bin/clang -fopenmp -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/opt/gettext/lib -L/usr/local/opt/llvm/lib -o data.table.so assign.o between.o bmerge.o chmatch.o dogroups.o fastmean.o fcast.o fmelt.o forder.o frank.o fread.o fsort.o fwrite.o gsumm.o ijoin.o init.o inrange.o openmp-utils.o quickselect.o rbindlist.o reorder.o shift.o subset.o transpose.o uniqlist.o vecseq.o wrappers.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
mv data.table.so datatable.so
if [ "" != "Windows_NT" ] && [ `uname -s` = 'Darwin' ]; then install_name_tool -id datatable.so datatable.so; fi
installing to /Users/rob/Library/R/3.3/library/data.table/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error in dyn.load(file, DLLpath = DLLpath, ...) :
  unable to load shared object '/Users/rob/Library/R/3.3/library/data.table/libs/datatable.so':
  dlopen(/Users/rob/Library/R/3.3/library/data.table/libs/datatable.so, 6): Library not loaded: @rpath/libomp.dylib
  Referenced from: /Users/rob/Library/R/3.3/library/data.table/libs/datatable.so
  Reason: image not found
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Users/rob/Library/R/3.3/library/data.table’

The downloaded source packages are in
    ‘/private/var/folders/vv/5j3h0wz17z1dg25swdnk23lr0000gn/T/RtmpIsZxRA/downloaded_packages’
Warning message:
In install.packages("data.table", type = "source", repos = "http://Rdatatable.github.io/data.table") :
  installation of package ‘data.table’ had non-zero exit status

R version:

> R.version
               _
platform       x86_64-apple-darwin13.4.0
arch           x86_64
os             darwin13.4.0
system         x86_64, darwin13.4.0
status
major          3
minor          3.3
year           2017
month          03
day            06
svn rev        72310
language       R
version.string R version 3.3.3 (2017-03-06)
nickname       Another Canoe

install openmp

Source

robocop-bob

Most helpful comment

I ran into this problem too :/

I was able to get it to work by switching to gcc (takes ~30 minutes to reinstall):

brew reinstall gcc --without-multilib

Then edit ~/.R/Makevars:

# GCC version:
VER=6
CC=gcc-${VER}
CXX=g++-${VER}
CXX1X=g++-${VER}

# Use 8 threads if you have a 4-core intel cpu. Could be -j2 or -j4 alternatively.
MAKE=make -j8

SHLIB_OPENMP_CFLAGS=-fopenmp
SHLIB_OPENMP_CXXFLAGS=-fopenmp
SHLIB_OPENMP_FCFLAGS=-fopenmp
SHLIB_OPENMP_FFLAGS=-fopenmp

Then install.packages("data.table") succeeded at least. I'm not certain if multithreading is working or not though.

Source: http://thecoatlessprofessor.com/programming/openmp-in-r-on-os-x/

ck37 on 16 Mar 2017

👍2

All 10 comments

I encountered same situation, but I didn't solve this problem, either.

R.version
_
platform x86_64-apple-darwin13.4.0
arch x86_64
os darwin13.4.0
system x86_64, darwin13.4.0
status
major 3
minor 3.2
year 2016
month 10
day 31
svn rev 71607
language R
version.string R version 3.3.2 (2016-10-31)
nickname Sincere Pumpkin Patch

mn-tk on 14 Mar 2017

I pasted the error into Google: "Library not loaded libomp.dylib"
Does this help? http://stackoverflow.com/questions/17703510/dyld-library-not-loaded-reason-image-not-loaded
It is not about the libomp library but it seems to be the same problem with a different library. Perhaps try following the same steps. If it works, please could you add something to the Installation wiki page.

mattdowle on 14 Mar 2017

👎1

I am sorry , I don't know how to use your advise in this situation :)

robocop-bob on 15 Mar 2017

👍2

I ran into this problem too :/

I was able to get it to work by switching to gcc (takes ~30 minutes to reinstall):

brew reinstall gcc --without-multilib

Then edit ~/.R/Makevars:

# GCC version:
VER=6
CC=gcc-${VER}
CXX=g++-${VER}
CXX1X=g++-${VER}

# Use 8 threads if you have a 4-core intel cpu. Could be -j2 or -j4 alternatively.
MAKE=make -j8

SHLIB_OPENMP_CFLAGS=-fopenmp
SHLIB_OPENMP_CXXFLAGS=-fopenmp
SHLIB_OPENMP_FCFLAGS=-fopenmp
SHLIB_OPENMP_FFLAGS=-fopenmp

Then install.packages("data.table") succeeded at least. I'm not certain if multithreading is working or not though.

Source: http://thecoatlessprofessor.com/programming/openmp-in-r-on-os-x/

ck37 on 16 Mar 2017

👍2

After reinstall gcc and changes in ~/.R/Makevars suggested by @ck37 - similar effect.
I had to run brew unlink llvm ... suggested by brew doctor.
After this step compilation works OK.
library(data.table) , not info regarding "...not detected OpenMP support..."

Regarding Wiki - I don't know if it is solution or just temporary work around ( last documentation shows clang , not gcc )

robocop-bob on 16 Mar 2017

👍1

I've had the same issue on two machines. @ck37 solution did the trick.

UPDATE: After installing data.table, it is important to change the Makevars file
I used:
CC=clang
CXX=clang++

Leaving it as is will create issues in other packages which might lead to segfaults.
For me, I installed Rpostgres with the settings above (with fopenmp), and every database query error would crash R. I then reinstalled Rpostgres using clang and no longer received segfaults. (errors were handled gracefully).

rsaporta on 17 Mar 2017

You need to use -rpath as well as -L. This is noted in the Homebrew caveat for LLVM now:

To use the bundled libc++ please add the following LDFLAGS:
  LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"

but this really applies to all "bundled" libraries, including OpenMP.

This is my ~/.R/Makevars file:

CC      = ccache /usr/local/opt/llvm/bin/clang
# CC      = ccache /usr/local/bin/gcc-6
CXX       = ccache /usr/local/opt/llvm/bin/clang++
# CXX       = ccache /usr/local/bin/g++-6
F77    = ccache /usr/local/bin/gfortran
FC     = ccache /usr/local/bin/gfortran

FFLAGS   += -O3 -pipe
FCLAGS   += $(FFLAGS)
CFLAGS   += -O3 -mtune=native -pipe -pedantic -Wall -Wno-unused-command-line-argument
CXXFLAGS += $(CFLAGS) 

SHLIB_OPENMP_CFLAGS   = -fopenmp
SHLIB_OPENMP_CXXFLAGS = -fopenmp
SHLIB_OPENMP_FCFLAGS  = -fopenmp
SHLIB_OPENMP_FFLAGS   = -fopenmp

LDFLAGS         += -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib
CPPFLAGS        += -L/usr/local/opt/llvm/include
PKG_CONFIG_PATH += $(PKG_CONFIG_PATH):/usr/local/opt/llvm/lib/pkgconfig

With this, install.packages("data.table") works just fine.

gwerbin on 25 Mar 2017

Using the Homebrew caveat as @gwerbin suggested to extend the Makevars from the Installation Wiki I tried this:

CC=/usr/local/opt/llvm/bin/clang -fopenmp
CXX=/usr/local/opt/llvm/bin/clang++
LDFLAGS=-L/usr/local/opt/gettext/lib -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib
CPPFLAGS=-I/usr/local/opt/gettext/include -I/usr/local/opt/llvm/include

It compiles w/o any errors and uses multithreading.

ktoth on 27 Mar 2017

👍1

Thanks @ktoth. Please write back if still an issue.

arunsrinivasan on 31 Mar 2017

install -->
sudo apt-get install pkg-config
and then try to install
install.packages("data.table", type = "source",repo="http://Rdatatable.github.io/data.table")