Alpaka: "Could NOT find Threads" during cmake build in mac

[kakwok]

Hi,

I am following the installation instructions from [1], but when I reach the step:

cmake -DCMAKE_INSTALL_PREFIX=/install/ ..

I get the following error:

cmake -DCMAKE_INSTALL_PREFIX=/install/ ..
-- The CXX compiler identification is Clang 4.0.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Users/kakwok/anaconda3/bin/x86_64-apple-darwin13.4.0-clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - no
CMake Error at /Applications/CMake.app/Contents/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:218 (message):
  Could NOT find Threads (missing: Threads_FOUND)
Call Stack (most recent call first):
  /Applications/CMake.app/Contents/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:577 (_FPHSA_FAILURE_MESSAGE)
  /Applications/CMake.app/Contents/share/cmake-3.19/Modules/FindThreads.cmake:234 (FIND_PACKAGE_HANDLE_STANDARD_ARGS)
  cmake/alpakaCommon.cmake:141 (find_package)
  CMakeLists.txt:77 (include)

System info:

cmake version 3.19.0-rc1
alpaka release-0.6.0-rc1
boost BOOST_LIB_VERSION "1_74" BOOST_VERSION 107400
MACOS 10.14.5, Macbook air 2015

============================
From googling the CMAKE error, I found a workaround by putting the following snippet in the CMakeLists.txt in the alpaka directory

set(CMAKE_THREAD_LIBS_INIT "-lpthread")
set(CMAKE_HAVE_THREADS_LIBRARY 1)
set(CMAKE_USE_WIN32_THREADS_INIT 0)
set(CMAKE_USE_PTHREADS_INIT 1)
set(THREADS_PREFER_PTHREAD_FLAG ON)

But I'm not sure if this is supposed to be the right way to configure the build

[1]https://alpaka.readthedocs.io/en/latest/basic/install.html#

[BenjaminW3]

@psychocoderHPC @j-stephan May you be able to help as you have been working on related parts in #866 ?

[psychocoderHPC]

@BenjaminW3 The issue #866 is different from this one. For some reason PThread at all can not be found. Where in #866 we try to solve the problem that we should set -pthread for the compile step if we use -lpthread during the linking.

@kakwok I do not have access to a MACOSX system so I can only guess. Is it possible that you install a modern clang compiler on your system? From the logs I see that you uses clang 4.0.1.

[psychocoderHPC]

@kakwok is the workaround

set(CMAKE_THREAD_LIBS_INIT "-lpthread")
set(CMAKE_HAVE_THREADS_LIBRARY 1)
set(CMAKE_USE_WIN32_THREADS_INIT 0)
set(CMAKE_USE_PTHREADS_INIT 1)
set(THREADS_PREFER_PTHREAD_FLAG ON)

working for you?
For me the workaround is bypassing the CMake detection.

Source of the workaround: https://github.com/alicevision/geogram/issues/2#issuecomment-536835320

[j-stephan]

Could you paste the contents of CMakeError.log and CMakeOutput.log? They should live in the CMakeFiles directory that was created by running cmake.

[kakwok]

@psychocoderHPC
Yes, I found the same page for that workaround and it worked for me. And yes, I think I am using a version of clang from anaconda as default, is that an issue?

@j-stephan Here're the log files:
CMakeError.log
CMakeOutput.log

[j-stephan]

Yes, I found the same page for that workaround and it worked for me. And yes, I think I am using a version of clang from anaconda as default, is that an issue?

I believe this may be the cause of your issues. Your CMakeError.log indicates that CMake is indeed picking up Anaconda's clang (which is ancient) but then tries to link to the system libraries - the latter presumably being built with much newer clang versions. This causes weird linker errors, in this case resulting in pthread being unlinkable.

This seems to be a common issue with Anaconda on macOS. By default Anaconda silently and permanently overrides important environment variables (CC, CXX, ...). That is a very questionable choice by the Anaconda developers. I suggest you file a bug report on their issue tracker.

You have two choices to fix this:

1. Permanently reset the environment variables to something intelligent (-> the compiler which is part of XCode) and point Anaconda to its own toolchain whenever you need it. I recommend this approach because otherwise you are likely to encounter this error with other projects.
2. Tell CMake on a case-by-case basis where to find the correct compiler: CXX=/path/to/less/ancient/compiler cmake [INSERT_OPTIONS_HERE]. Use this if you are not planning to compile anything else soon.

[psychocoderHPC]

@kakwok The best way to avoid this issue is to have a separate environment without anaconda to build and work with alpaka and an environment to post-process stuff with python where you load you anaconda environment. As @j-stephan wrote, anaconda is often introducing side effects.

[j-stephan]

@kakwok Is this still an issue on your side?