Detectron2: Can't `FORCE_CUDA=1 pip install -e .` detectron2 during docker build on a host without a GPU

[guy4261]

How To Reproduce the Issue

I'm building a Dockerfile on my OS X machine (that has no GPU).. I'm using standard Docker, but the image I'm building will run on a cluster machine with an NVIDIA GPU and the CUDA driver (where nvidia-docker is used).

Inside the building container, after the CUDA-10.0 toolkit has been installed successfully and CUDA_HOME has been set, one of the RUN commands in the Dockerfile is:

RUN FORCE_CUDA=1 pip install -e .` on the `detectron2

Actual results

This pip install for detecron2 returns:

RuntimeError: cuda runtime error (38) : no CUDA-capable device is detected at /pytorch/aten/src/THC/THCGeneral.cpp:50

Expected behavior

I think my use case is fairly reasonable - build a docker image suitable for CUDA/GPU, on a machine that does not have it. But it fails.

If I don't try to install detectron2 but instead run the image on the said cluster and install there, the installation works great and the demo also runs properly.

I'd like to be able to build for CUDA even if a CUDA device is not present.

Environment

root@8e9a88a907bf:/detectron2/detectron2/utils# python collect_env.py
---------------------  ------------------------------------------------------------------------------
sys.platform           linux
Python                 3.6.7 | packaged by conda-forge | (default, Nov  6 2019, 16:19:42) [GCC 7.3.0]
Numpy                  1.16.4
detectron2._C          failed to import
DETECTRON2_ENV_MODULE  <not set>
PyTorch                1.3.0+cu100
PyTorch Debug Build    False
torchvision            0.4.1+cu100
CUDA available         False
Pillow                 6.2.1
cv2                    4.1.0
---------------------  ------------------------------------------------------------------------------
PyTorch built with:
  - GCC 7.3
  - Intel(R) Math Kernel Library Version 2019.0.4 Product Build 20190411 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v0.20.5 (Git Hash 0125f28c61c1f822fd48570b4c1066f96fcb9b2e)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - NNPACK is enabled
  - Build settings: BLAS=MKL, BUILD_NAMEDTENSOR=OFF, BUILD_TYPE=Release, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -fopenmp -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Wno-stringop-overflow, DISABLE_NUMA=1, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, USE_CUDA=True, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_STATIC_DISPATCH=OFF,

[ppwwyyxx]

RuntimeError: cuda runtime error (38) : no CUDA-capable device is detected at /pytorch/aten/src/THC/THCGeneral.cpp:50

It is unclear what command you run on what machine when you saw this error.
The error is just saying that no gpu is detected and you need to check that. If you can't figure it out, please provide your environment information on that machine following the issue template.

[guy4261]

Thanks @ppwwyyxx . Re-edited the issue description according to the template.

[ppwwyyxx]

It would be always better to post full logs than less informative partial logs.
My guess is you need https://github.com/facebookresearch/detectron2/blob/cd3a0f9bff06f8cf13552099504213f0d057c7e1/Dockerfile#L26
and in any case it's good to start from what we provided.

[guy4261]

Perfect guess - that worked! Thanks a million!