Data.table: Error in MacOS "Assertion failure at kmp_runtime.cpp(6480)"

[dracodoc]

This issue was reported earlier in #2418, created as separate issue per @mattdowle 's request.

Assertion failure at kmp_runtime.cpp(6480): __kmp_thread_pool == __null.
Assertion failure at kmp_runtime.cpp(6480): __kmp_thread_pool == __null.
OMP: Error #13: Assertion failure at kmp_runtime.cpp(6480).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see http://www.intel.com/software/products/support/.
OMP: Error #13: Assertion failure at kmp_runtime.cpp(6480).
OMP: Hint: Please submit a bug report with this message, compile and run commands used, and machine configuration info including native compiler and operating system versions. Faster response will be obtained by including all program sources. For information on submitting this issue, please see http://www.intel.com/software/products/support/.
Assertion failure at kmp_runtime.cpp(6480): __kmp_thread_pool == __null.
Assertion failure at kmp_runtime.cpp(6480): __kmp_thread_pool == __null.

I just upgraded to dev 1.10.5 and still have this problem:

• This error usually appear randomly in my parallel code (used parallel package, used data.table indirectly, i.e. the parallel code doesn't use data.table but the environment have data.table loaded).
• It appear regularly in knitting a vignette. Running all chunks from same vignette don't have this error.
• Add a line of mclapply(1:2, sqrt) in the beginning of the vignette can make the error disappear.
• @iivoi reported similar problem.

I have my vignettes linked above, you can run it to test the error after installing my package ctmmweb from the github page.

> sessionInfo()
R version 3.4.4 (2018-03-15)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.2

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] ctmmweb_0.1.0.9000 ctmm_0.5.0        

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.16      magrittr_1.5      knitr_1.20        raster_2.6-7      munsell_0.4.3    
 [6] colorspace_1.3-2  lattice_0.20-35   rlang_0.2.0       stringr_1.3.0     plyr_1.8.4       
[11] tools_3.4.4       grid_3.4.4        data.table_1.10.5 gtable_0.2.0      htmltools_0.3.6  
[16] yaml_2.1.18       lazyeval_0.2.1    digest_0.6.15     rprojroot_1.3-2   tibble_1.4.2     
[21] ggplot2_2.2.1     htmlwidgets_1.0   evaluate_0.10.1   rmarkdown_1.9     sp_1.2-7         
[26] stringi_1.1.7     compiler_3.4.4    pillar_1.2.1      scales_0.5.0      backports_1.1.2  

[ghost]

I managed to get dev 1.10.5 running with OpenMP. I think the key might have been a brew version of the libomp, though this is not within my area of expertise anymore. Followed the instructions below (copied from stackoverflow.com):

mkdir omp_clang && cd omp_clang

git clone https://github.com/llvm-mirror/llvm.git -b release_60
git clone https://github.com/llvm-mirror/clang.git llvm/tools/clang -b release_60

mkdir build && cd build
cmake ../llvm
make

brew install libomp

And then in R:

remove.packages("data.table")
install.packages("data.table", type = "source",
    repos = "http://Rdatatable.github.io/data.table")

Hope this helps, both as a quick fix and as something towards a more robust solution. Maybe @dracodoc give it a go? The only other difference that I see with your report above is that I'm on MacOS 10.13.4.

[dracodoc]

I also get OpenMP working (see this) but the error is still there.

The OpenMP discussion in previous issue is more related to the problem "OpenMP is not supported in certain MacOS so data.table cannot be installed from source, compiled with OpenMP". That problem can be solved with either enable OpenMP, or just disable OpenMP compiling all together.

Did dev 1.10.5, openMP solved problem for you? I have delayed the MacOS upgrade(I only see 10.12.6 update in app store for my system) because I didn't have any problem before and don't want to introduce new problems.

[ghost]

Yes, everything seems to be ok here following the tricks above. Not sure if installing clang is required, but I did it anyhow. Then libomp from brew, removed data.table from R and reinstalled dev-version. Prior to these tricks it was working randomly, same as you.

On 25 Apr 2018, at 22.26, dracodoc [e-mail] wrote:

I also get OpenMP working (see this https://github.com/Rdatatable/data.table/issues/2418#issuecomment-340010541) but the error is still there.

The OpenMP discussion in previous issue is more related to the problem "OpenMP is not supported in certain MacOS so data.table cannot be installed from source, compiled with OpenMP". That problem can be solved with either enable OpenMP, or just disable OpenMP compiling all together.

Did dev 1.10.5, openMP solved problem for you? I have delayed the MacOS upgrade because I didn't have any problem before and don't want to introduce new problems.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/Rdatatable/data.table/issues/2787#issuecomment-384405604, or mute the thread https://github.com/notifications/unsubscribe-auth/AjezzvXC1dGyYPa2q6Doc9IXTW--FS1sks5tsM3jgaJpZM4Tf3eM.

[dracodoc]

I upgraded MacOS to 10.12.6 (this is the highest version available to my system, it's a 2015 macbook), installed libomp through brew, installed data.table dev 1.10.5.

However the error is still there.

[jangorecki]

@dracodoc if you manage to solve your problem you can close this issue. Consider putting relevant information to Installation wiki.

[dracodoc]

No the problem is not solved. Maybe my previous comment is not clear enough, I'll edit it.

[dracodoc]

I upgraded to R 3.5 and problem solved!

The stuff I tried:

1. I installed gfortran and clang prebuilt from here,
2. upgraded MacOS to 10.12.6, the clang version seemed to be 8.0.0. This probably override the clang in step 1, but the makdvar settings from step 1 might be useful.
3. install R 3.5. Lots of packages need to be reinstalled, data.table was installed with CRAN version.