Theano: macOS 10.12 CUDA 8.0: Clang not supported / Failed to compile cuda_ndarray.cu

[subzerofun]

Hello,
i´ve used theano with OSX 10.11 and my GTX 780 without much problems. After upgrading to 10.12 i decided to update my CUDA Version to 8.0 and reinstall theano. The CUDA driver is up-to-date. I use XCode 8.1 and the latest Command Line Tools.

When i try to run the GPU test (from the tutorial) i get the following error messages:
Part 1 are hundreds of pages of code... http://pastebin.com/M6kaGQvH (didn´t want to spam github)
And Part 2 is maybe more telling:

===============================
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
nvcc fatal   : The version ('80000') of the host compiler ('Apple clang') is not supported

['nvcc', '-shared', '-O3', '-use_fast_math', '--compiler-bindir', '/usr/bin/clang', '-m64', '-Xcompiler', '-DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden', '-Xlinker', '-rpath,/Users/david/.theano/compiledir_Darwin-16.0.0-x86_64-i386-64bit-i386-3.5.2-64/cuda_ndarray', '-Xlinker', '-rpath,/usr/local/cuda/lib', '-I/usr/local/lib/python3.5/site-packages/theano/sandbox/cuda', '-I/usr/local/lib/python3.5/site-packages/numpy/core/include', '-I/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/include/python3.5m', '-I/usr/local/lib/python3.5/site-packages/theano/gof', '-L/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib', '-o', '/Users/david/.theano/compiledir_Darwin-16.0.0-x86_64-i386-64bit-i386-3.5.2-64/cuda_ndarray/cuda_ndarray.so', 'mod.cu', '-lcublas', '-lcudart', '-Xcompiler', '-undefined,dynamic_lookup', '-Xlinker', '-pie']
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: ('nvcc return status', 1, 'for cmd', 'nvcc -shared -O3 -use_fast_math --compiler-bindir /usr/bin/clang -m64 -Xcompiler -DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker -rpath,/Users/david/.theano/compiledir_Darwin-16.0.0-x86_64-i386-64bit-i386-3.5.2-64/cuda_ndarray -Xlinker -rpath,/usr/local/cuda/lib -I/usr/local/lib/python3.5/site-packages/theano/sandbox/cuda -I/usr/local/lib/python3.5/site-packages/numpy/core/include -I/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/include/python3.5m -I/usr/local/lib/python3.5/site-packages/theano/gof -L/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib -o /Users/david/.theano/compiledir_Darwin-16.0.0-x86_64-i386-64bit-i386-3.5.2-64/cuda_ndarray/cuda_ndarray.so mod.cu -lcublas -lcudart -Xcompiler -undefined,dynamic_lookup -Xlinker -pie')
Traceback (most recent call last):
  File "test_theano.py", line 1, in <module>
    from theano import function, config, shared, sandbox
  File "/usr/local/lib/python3.5/site-packages/theano/__init__.py", line 107, in <module>
    import theano.sandbox.cuda
  File "/usr/local/lib/python3.5/site-packages/theano/sandbox/cuda/__init__.py", line 713, in <module>
    use(device=config.device, force=config.force_device, test_driver=False)
  File "/usr/local/lib/python3.5/site-packages/theano/sandbox/cuda/__init__.py", line 503, in use
    cuda_initialization_error_message))
OSError: You forced the use of gpu device gpu, but CUDA initialization failed with error:
cuda unavailable

So these two stand out:

nvcc fatal : The version ('80000') of the host compiler ('Apple clang') is not supported

ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu:

This is my .theanorc file:

[global]
floatX = float32
device = gpu
init_gpu_device = gpu
force_device = True
allow_gc= True
print_active_device = True
enable_initial_driver_test = False

[gcc]
cxxflags = -L/usr/local/lib -L/Developer/NVIDIA/CUDA-8.0/lib/
#cxxflags = -L/usr/local/lib -L/Developer/NVIDIA/CUDA-7.5/lib/

[nvcc]
fastmath = True
compiler_bindir=/usr/bin/clang
optimizer_including=cudnn

[cuda]
root = /Developer/NVIDIA/CUDA-8.0
#root = /Developer/NVIDIA/CUDA-7.5

[blas]
#ldflags = -L/usr/local/Cellar/openblas -lopenblas

[lib]
#cnmem=.5

Do i need to revert to an older version of clang and CUDA?

[kashif]

please update your cuda toolkit, the new version now supports xcode 8

[leocnj]

Using

clang 8.1.0 with XCode 8.3
CUDA 8.0.64

met the error like

WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available (error: cuda unavailable)

It is very strange that GPU version theano never works after my OS update to 10.12.

[kashif]

@leocnj because nvcc is hard-coded to only work with certain clang version...

[leocnj]

nvcc

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:46_CST_2017
Cuda compilation tools, release 8.0, V8.0.61

clang

clang --version
Apple LLVM version 8.1.0 (clang-802.0.38)
Target: x86_64-apple-darwin16.4.0
Thread model: posix
InstalledDir: >/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

@kashif, thanks. Do you think that the above pair will have a chance to run on my OS 10.12 or not. Note that this only impacts theano. tensorflow GPU runs well.

[kashif]

@leocnj no it will not work nvidia need to release an updated cuda toolkit for this Xcode...

[Sentient07]

I use a macbook running macOS Sierra 10.12. In my experience, X Code 8.x is not compatible with cuda 8.0 toolkit. Before it used to be XCode 6.x working with cuda 7.5, now XCode 7.x works with cuda 8.0.

[leocnj]

@kashif and @Sentient07 thanks for your answers. It looks that I need downgrade my XCode to 7.x. Glad to know the reason of this weird issue bothering me for months.

[Sentient07]

A quick correction to my previous answer. It seems like a couple of months later from writing a post on theano-dev, Nvidia has added support for LLVM 8.0. But make sure you are using a version of cuda that is after 12/12/2016. A table summarising the requirements, link