Data.table: support OpenMP with system macOS toolchain

[kevinushey]

With R 4.0.0, R (as well as packages on CRAN) will be built with the system toolchain (Apple Clang), which unfortunately mean OpenMP support is not provided natively by the compiler. Fortunately, it is still possible to use OpenMP with the system toolchain if you:

1. Instruct clang to enable OpenMP with -Xpreprocessor -fopenmp,
2. Explicitly provide include paths to libomp during compilation,
3. Explicitly provide library paths to libomp during link.

Unfortunately, data.table fails to detect this case as it invokes the compiler with just -fopenmp here:

https://github.com/Rdatatable/data.table/blob/b1b1832b0d2d4032b46477d9fe6efb15006664f4/configure#L64

It would be helpful for users on macOS if support for OpenMP on macOS could be detected + used.

Sorry, I recognize this is kind of a pain but figured it was worth filing since OpenMP is somewhat vital to data.table's performance.

See also:

http://r-sig-mac.29524.n8.nabble.com/R-SIG-Mac-Apple-Clang-does-support-OpenMP-if-libomp-is-available-td458.html#a461

(+ some other recent discussion on R-SIG-Mac)

[MichaelChirico]

I think Jim Hester raised this same issue, let me see...

I see, it was on the orgs page... good to have this in the issue tracker

https://github.com/orgs/Rdatatable/teams/project-members/discussions/34

[krivard]

I am pasting my working version of Makevars here with edits from the version in the install docs marked, so that google may find it and help a future traveller:

# Added: -Xpreprocessor
CC=/usr/bin/clang -Xpreprocessor -fopenmp

# Added: -Xpreprocessor -lomp -I/usr/local/include
CXX=/usr/bin/clang++ -Xpreprocessor -fopenmp -lomp -I/usr/local/include
# -O3 should be faster than -O2 (default) level optimisation ..
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe

# Added: -lomp
LDFLAGS=-L/usr/local/opt/gettext/lib -L/usr/lib -Wl,-rpath,/usr/lib -lomp
CPPFLAGS=-I/usr/local/opt/gettext/include -I/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include

macOS Mojave, 10.14.6

with thanks to these two threads on StackOverflow

[dbrookeUAB]

Since @krivard last post was super helpful for me to get things working, here is my working version of Makevars

# 2020-11-15 Added:  -Xpreprocessor -fopenmp 
CC=/usr/local/Cellar/llvm/11.0.0/bin/clang -Xpreprocessor -fopenmp
# 2020-11-15 Added:   -Xpreprocessor -fopenmp -lomp -I/usr/local/include
CXX=/usr/local/Cellar/llvm/11.0.0/bin/clang++  -Xpreprocessor -fopenmp -lomp -I/usr/local/include
CXX11=/usr/local/Cellar/llvm/11.0.0/bin/clang++
CXX14=/usr/local/Cellar/llvm/11.0.0/bin/clang++
CXX17=/usr/local/Cellar/llvm/11.0.0/bin/clang++
CXX1X=/usr/local/Cellar/llvm/11.0.0/bin/clang++
# 2020-11-15 Added:  CFLAGS
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
# 2020-11-15 Added:  CXXFLAGS
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L/usr/local/Cellar/llvm/11.0.0/lib
SHLIB_OPENMP_CFLAGS= -fopenmp
SHLIB_OPENMP_CXXFLAGS= -fopenmp

Also Mac Mojave 10.14.6

Please reply if you find anything about this that could be improved or if you find something redundant. I just change lines on a whim because I'm too lazy to spend the time really learning the ins and outs of the Makevars file.

[jangorecki]

@kevinushey could you please check if 1.13.2 resolves the problem for you? There was a change in https://github.com/Rdatatable/data.table/pull/4735

1. User-supplied PKG_LIBS and PKG_CFLAGS are now retained and the suggestion in https://mac.r-project.org/openmp/; i.e.,
   PKG_CPPFLAGS='-Xclang -fopenmp' PKG_LIBS=-lomp R CMD INSTALL data.table_<ver>.tar.gz
   has a better chance of working on Mac.

[kevinushey]

Looks like this works for me as well. It sounds like the issue is ultimately resolved, then? Unless data.table wants to try and automagically enable OpenMP on macOS.

[mattdowle]

@kevinushey Great to hear it works.

Unless data.table wants to try and automagically enable OpenMP on macOS.

That's an interesting idea. I don't have macOS and was waiting to see how macOS users got on with the latest version. I got the impression from https://mac.r-project.org/openmp/ that there are so many problems with macOS and OpenMP that there is not really much we can do. But, now that you said this, perhaps we could have another go. That would mean that the CRAN binary would be OpenMP-enabled again, iiuc? That would be great. If so, what exactly works for you: simply passing -Xclang -fopenmp and that's all that's needed?

[kevinushey]

Right, and I can see arguments both ways:

1. On one hand, the use of OpenMP with Apple Clang isn't officially supported, and so for that reason data.table shouldn't try to enable it by default (since in theory Apple could pull the rug out from underneath us at any time);

2. On the other hands, as far as I know no users have reported issues when trying to use OpenMP with Apple Clang + data.table anyhow, so even if it's not _officially_ supported it's still functional as far as it's used in data.table, so perhaps it's worth (trying to) enable by default.

I don't have a strong opinion on what the best way forward is, though.

[mattdowle]

If data.table makes an attempt in the configure script, and performs the attempt correctly by falling back to no OpenMP, then it looks like we're agreeing on option 2 then. It's the performing-the-attempt-correctly part that is tricky cross-platform. Did you see the comment I put into configure -- I'm not sure that attempt (testing to see if manually adding -fopenmp works) has ever worked as intended since it was put in a few versions ago.

[kevinushey]

I think you're right (sorry about that). It did ultimately work insofar that OpenMP was disabled by default on macOS, since the "default" OpenMP flags wouldn't work there. But it didn't help in turning OpenMP on.

Regardless, I think getting this right would mean redoing that part of the configure script, to keep track of + set the OpenMP compiler flags as appropriate in the Makevars file.