Vcpkg: [cuda:x64-linux] vcpkg cannot find CUDA SDK on Ubuntu

Created on 19 Sep 2019 · 10Comments · Source: microsoft/vcpkg

vcpkg cannot find CUDA SDK on Ubuntu, despite it is installed from official CUDA .deb package.
This happens regardless if one version of CUDA is installed (CUDA 10.1 Update 2), or many versions at the same time (CUDA 9, 10, 10.1 Update 2).

Host Environment

OS: Ubuntu 16.04 and Ubuntu 18.04
Compiler: N/A

To Reproduce
Steps to reproduce the behavior:
./vcpkg install cuda

Failure logs

$ ./vcpkg install cuda
The following packages will be built and installed:
    cuda[core]:x64-linux
Starting package 1/1: cuda:x64-linux
Building package cuda[core]:x64-linux...
CMake Error at ports/cuda/portfile.cmake:26 (message):
  Could not find CUDA.  Before continuing, please download and install CUDA
  (V9.0.0 or higher) from:

      https://developer.nvidia.com/cuda-downloads



  Also ensure vcpkg has been rebuilt with the latest version (v0.0.104 or
  later)
Call Stack (most recent call first):
  scripts/ports.cmake:91 (include)


Error: Building package cuda:x64-linux failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: cuda:x64-linux
  Vcpkg version: 2019.09.12-unknownhash

Additionally, attach any relevant sections from the log files above.

Additional context
Same bug on Windows was fixed in this issue: #3609

port-bug

Source

ZbigniewRA

Most helpful comment

To make the port work, the environment variable CUDA_PATH and CUDA_BIN_PATH are required.

Try this.

/path/to/vcpkg $ export CUDA_PATH=/usr/local/cuda;
/path/to/vcpkg $ export CUDA_BIN_PATH=/usr/local/cuda/bin;
/path/to/vcpkg $ vcpkg install cuda;
...

For Windows and PowerShell users, the solution is exactly the same with the above.

PS C:\Path\To\CUDA> $env:CUDA_PATH=Get-Location
PS C:\Path\To\CUDA\bin> $env:CUDA_PATH=Get-Location
PS C:\Path\To\vcpkg> ./vcpkg.exe install --triplet x64-windows cuda

Of course, you can use the string type instead of Get-Location command.

PS C:\Path\To\vcpkg> $env:CUDA_PATH="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1"

luncliff on 16 Oct 2019

👍3 🚀1 🎉1

All 10 comments

Any update or workaround on this issue? I have the same problem.

zentjosh on 3 Oct 2019

To make the port work, the environment variable CUDA_PATH and CUDA_BIN_PATH are required.

Try this.

/path/to/vcpkg $ export CUDA_PATH=/usr/local/cuda;
/path/to/vcpkg $ export CUDA_BIN_PATH=/usr/local/cuda/bin;
/path/to/vcpkg $ vcpkg install cuda;
...

For Windows and PowerShell users, the solution is exactly the same with the above.

PS C:\Path\To\CUDA> $env:CUDA_PATH=Get-Location
PS C:\Path\To\CUDA\bin> $env:CUDA_PATH=Get-Location
PS C:\Path\To\vcpkg> ./vcpkg.exe install --triplet x64-windows cuda

Of course, you can use the string type instead of Get-Location command.

PS C:\Path\To\vcpkg> $env:CUDA_PATH="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1"

luncliff on 16 Oct 2019

👍3 🚀1 🎉1

To make the port work, the environment variable CUDA_PATH and CUDA_BIN_PATH are required.

Try this.
/path/to/vcpkg $ export CUDA_PATH=/usr/local/cuda;
/path/to/vcpkg $ export CUDA_BIN_PATH=/usr/local/cuda/bin;
/path/to/vcpkg $ vcpkg install cuda;
...
For Windows and PowerShell users, the solution is exactly the same with the above.
PS C:\Path\To\CUDA> $env:CUDA_PATH=Get-Location
PS C:\Path\To\CUDA\bin> $env:CUDA_PATH=Get-Location
PS C:\Path\To\vcpkg> ./vcpkg.exe install --triplet x64-windows cuda
Of course, you can use the string type instead of Get-Location command.
PS C:\Path\To\vcpkg> $env:CUDA_PATH="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1"

I set the CUDA_PATH and CUDA_BIN_PATH in command line. It works well

xds95 on 10 Apr 2020

I will handle this issue in next week.

JackBoosY on 10 Apr 2020

Hi guys, I'm fixing this issue in #10838, could anyone help me to test it?

Thanks.

JackBoosY on 16 Apr 2020

🚀1

Just checked #10838 with the Docker image, nvidia/cuda.

The image doesn't have environment variable CUDA_PATH and CUDA_BIN_PATH, but has nvcc in its PATH.

$ docker inspect --format "{{ json .Config.Env }}" nvidia/cuda:10.2-devel-ubuntu18.04 | jq
[
  "PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
  "CUDA_VERSION=10.2.89",
  "CUDA_PKG_VERSION=10-2=10.2.89-1",
  "LD_LIBRARY_PATH=/usr/local/nvidia/lib:/usr/local/nvidia/lib64",
  "NVIDIA_VISIBLE_DEVICES=all",
  "NVIDIA_DRIVER_CAPABILITIES=compute,utility",
  "NVIDIA_REQUIRE_CUDA=cuda>=10.2 brand=tesla,driver>=384,driver<385 brand=tesla,driver>=396,driver<397 brand=tesla,driver>=410,driver<411 brand=tesla,driver>=418,driver<419",
  "NCCL_VERSION=2.5.6",
  "LIBRARY_PATH=/usr/local/cuda/lib64/stubs"
]

Here is my reproduction step and its log.

FROM    nvidia/cuda:10.2-devel-ubuntu18.04

RUN     apt-get update -y -qq && apt-get install -y -qq --fix-missing \
        git g++-8 unzip tar curl

RUN     git clone --branch=dev/jack/8247 --depth=10 https://github.com/JackBoosY/vcpkg /vcpkg
WORKDIR /vcpkg
RUN     bash ./bootstrap-vcpkg.sh
RUN     ./vcpkg install cuda

Step 6/6 : RUN     ./vcpkg install cuda
 ---> Running in 2abd60d0ff33
Computing installation plan...
The following packages will be built and installed:
    cuda[core]:x64-linux
Starting package 1/1: cuda:x64-linux
Building package cuda[core]:x64-linux...
FOUND_PATH : /usr/local/cuda-10.2
-- Found CUDA V10.2.89
-- Performing post-build validation
-- Performing post-build validation done
Building package cuda[core]:x64-linux... done
Installing package cuda[core]:x64-linux...
Installing package cuda[core]:x64-linux... done
Elapsed time for package cuda:x64-linux: 108.1 ms

Total elapsed time: 109.6 ms

Removing intermediate container 2abd60d0ff33

luncliff on 16 Apr 2020

@luncliff Using environment variables is an unstable way. My modifications are based on FindCUDA.cmake provided by cmake.

JackBoosY on 20 Apr 2020

@JackBoosY Indeed it works well 👍

Removed NVIDIA related paths in PATH, but there was no problem.

RUN     bash ./bootstrap-vcpkg.sh

# ... there is no PATH for NVIDIA, CUDA ...
ENV     PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

RUN     ./vcpkg install cuda

luncliff on 21 Apr 2020

I upgraded my system to 20.04 LTS and reinstall CUDA toolkit via sudo apt install nvidia-cuda-toolkit, version 10.1.
Before ./vcpkg install cuda, export CUDA_PATH and CUDA_BIN_PATH environment variables:

$ export CUDA_PATH=/usr/lib/x86_64-linux-gnu/
$ export CUDA_BIN_PATH=/usr/bin

,then you will get successful info like:

$ ./vcpkg install cuda
Computing installation plan...
The following packages will be built and installed:
cuda[core]:x64-linux
Starting package 1/1: cuda:x64-linux
Building package cuda[core]:x64-linux...
-- Found CUDA V10.1.243
-- Performing post-build validation
-- Performing post-build validation done
Building package cuda[core]:x64-linux... done
Installing package cuda[core]:x64-linux...
Installing package cuda[core]:x64-linux... done
Elapsed time for package cuda:x64-linux: 13.9 ms

Total elapsed time: 14.12 ms