Cudf: [BUG] "library nvvm not found" when trying to run py.test

Created on 28 May 2019  路  2Comments  路  Source: rapidsai/cudf

If I:

  1. Build from source (v0.8),
  2. Install to $CONDA_PREFIX
  3. Run python setup.py build_ext --inplace
  4. Run py.test -v

I get library nvvm not found error (full trace below). I'm using Devuan ASCII (~= Debian Stretch), with CUDA 9.2 installed at /usr/local/cuda/ - and nvvm is right there. I made sure and added the relevant entries for it to LD_LIBRARY_PATH, LIBRARY_PATH, CPLUSINCLUDE_PATH etc.

This should not happen, in the sense that something should have failed earlier: The C++ build, the installation, the Python setup - I should get an error at this stage (or at all).

Error trace:

_____________________________________________________ ERROR collecting python/cudf/tests/test_dataframe.py _____________________________________________________
ImportError while importing test module '/mnt/sdb3/home-eyalroz/src/blazing/cudf/python/cudf/tests/test_dataframe.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
python/cudf/dataframe/columnops.py:446: in as_column
    data = as_column(memoryview(arbitrary))
E   TypeError: memoryview: a bytes-like object is required, not 'range'

During handling of the above exception, another exception occurred:
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/cudadrv/nvvm.py:111: in __new__
    inst.driver = open_cudalib('nvvm', ccc=True)
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/cudadrv/libs.py:48: in open_cudalib
    raise OSError('library %s not found' % lib)
E   OSError: library nvvm not found

During handling of the above exception, another exception occurred:
python/cudf/tests/test_dataframe.py:124: in <module>
    (Series(range(12)), Series(range(3, 12)), Series(range(0, 9, 2))),
python/cudf/dataframe/series.py:85: in __init__
    dtype=dtype)
python/cudf/dataframe/columnops.py:461: in as_column
    nan_as_null=nan_as_null
python/cudf/dataframe/columnops.py:392: in as_column
    pamask, padata = buffers_from_pyarrow(arbitrary)
python/cudf/utils/utils.py:156: in buffers_from_pyarrow
    copy_array(arrow_dev_array, mask_dev_array)
python/cudf/utils/cudautils.py:132: in copy_array
    gpu_copy.forall(arr.size)(arr, out)
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/compiler.py:226: in __call__
    kernel = self.kernel.specialize(*args)
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/compiler.py:776: in specialize
    kernel = self.compile(argtypes)
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/compiler.py:795: in compile
    kernel.bind()
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/compiler.py:503: in bind
    self._func.get()
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/compiler.py:381: in get
    ptx = self.ptx.get()
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/compiler.py:352: in get
    **self._extra_options)
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/cudadrv/nvvm.py:474: in llvm_to_ptx
    cu = CompilationUnit()
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/cudadrv/nvvm.py:146: in __init__
    self.driver = NVVM()
/home/eyalroz/miniconda2/envs/cudf/lib/python3.7/site-packages/numba/cuda/cudadrv/nvvm.py:116: in __new__
    raise NvvmSupportError(errmsg % e)
E   numba.cuda.cudadrv.error.NvvmSupportError: libNVVM cannot be found. Do `conda install cudatoolkit`:
E   library nvvm not found
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================== 1 skipped, 1 error in 2.09 seconds ==============================================================
? - Needs Triage bug

Most helpful comment

Did you set the required NUMBAPRO environment variables?
https://github.com/rapidsai/cudf/blob/feec0c5e21fef4a416c77656d1d3f43205aeddc1/CONTRIBUTING.md#build-from-source

You will also need the following environment variables, including $CUDA_HOME.
NUMBAPRO_NVVM=$CUDA_HOME/nvvm/lib64/libnvvm.so
NUMBAPRO_LIBDEVICE=$CUDA_HOME/nvvm/libdevice

All 2 comments

Did you set the required NUMBAPRO environment variables?
https://github.com/rapidsai/cudf/blob/feec0c5e21fef4a416c77656d1d3f43205aeddc1/CONTRIBUTING.md#build-from-source

You will also need the following environment variables, including $CUDA_HOME.
NUMBAPRO_NVVM=$CUDA_HOME/nvvm/lib64/libnvvm.so
NUMBAPRO_LIBDEVICE=$CUDA_HOME/nvvm/libdevice

I did not. That resolves it.

Was this page helpful?
0 / 5 - 0 ratings