Incubator-mxnet: Failure to build for Windows (CUDA or MKL-DNN)

[aluo-x]

Currently attempting to build on Windows 1803 with visual studio 2017 (15.6.7).

Also present on the device is CUDA 9.2.88 and 9.2.88.1, along with cuDNN 7.1.4, OpenCV 3.4.1, MKL 2018.3.210.

Other variables:

CUDA_PATH = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2

PATH = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\bin; C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\libnvvp; C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2

Steps to reproduce:

setx OpenCV_DIR "C:\Users\ALuo\Downloads\opencv\build\x64\vc15" 
setx MKL_ROOT "C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018\windows\mkl"
git clone https://github.com/apache/incubator-mxnet.git --recursive
cd incubator-mxnet
mkdir build
cmake -G "Visual Studio 15 Win64" .. -DUSE_CUDA=1 -DUSE_CUDNN=1 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=mkl -DUSE_LAPACK=0 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_MKLDNN=1 -DCMAKE_BUILD_TYPE=Release

Output:

-- Selecting Windows SDK version 10.0.16299.0 to target Windows 10.0.17134.
-- CMAKE_SYSTEM_NAME Windows
-- CMake version '3.11.4' using generator 'Visual Studio 15 2017 Win64'
CMake Error at C:/Program Files/CMake/share/cmake-3.11/Modules/CMakeDetermineCompilerId.cmake:251 (message):
  No CUDA toolset found.
Call Stack (most recent call first):
  C:/Program Files/CMake/share/cmake-3.11/Modules/CMakeDetermineCompilerId.cmake:31 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
  C:/Program Files/CMake/share/cmake-3.11/Modules/CMakeDetermineCUDACompiler.cmake:73 (CMAKE_DETERMINE_COMPILER_ID)
  CMakeLists.txt:54 (project)


-- Configuring incomplete, errors occurred!

Attempting to build with CUDA disabled:

cmake -G "Visual Studio 15 Win64" .. -DUSE_CUDA=0 -DUSE_CUDNN=0 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=0 -DUSE_PROFILER=1 -DUSE_BLAS=mkl -DUSE_LAPACK=0 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=All -DUSE_MKLDNN=1 -DCMAKE_BUILD_TYPE=Release

Output:

-- Selecting Windows SDK version 10.0.16299.0 to target Windows 10.0.17134.
-- CMAKE_SYSTEM_NAME Windows
F16C instruction set is not yet supported for MSVC
-- download mklml
-- [download 100% complete]

7-Zip [64] 15.14 : Copyright (c) 1999-2015 Igor Pavlov : 2015-12-31

Scanning the drive for archives:
1 file, 41454914 bytes (40 MiB)

Extracting archive: C:\Users\ALuo\Downloads\incubator-mxnet\build\mklml\mklml_win_2018.0.3.20180406.zip
--
Path = C:\Users\ALuo\Downloads\incubator-mxnet\build\mklml\mklml_win_2018.0.3.20180406.zip
Type = zip
Physical Size = 41454914

Everything is Ok

Folders: 3
Files: 25
Size:       155290955
Compressed: 41454914
CMake Deprecation Warning at 3rdparty/mkldnn/CMakeLists.txt:21 (cmake_policy):
  The OLD behavior for policy CMP0048 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


CMake Deprecation Warning at 3rdparty/mkldnn/CMakeLists.txt:22 (cmake_policy):
  The OLD behavior for policy CMP0054 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.


-- Detecting Intel(R) MKL: trying mklml_intel
-- Detecting Intel(R) MKL: trying mklml
-- Intel(R) MKL: include C:/Users/ALuo/Downloads/incubator-mxnet/build/mklml/mklml_win_2018.0.3.20180406/include
-- Intel(R) MKL: lib C:/Users/ALuo/Downloads/incubator-mxnet/build/mklml/mklml_win_2018.0.3.20180406/lib/mklml.lib
-- Intel(R) MKL: OpenMP lib C:/Users/ALuo/Downloads/incubator-mxnet/build/mklml/mklml_win_2018.0.3.20180406/lib/libiomp5md.lib
-- Intel(R) MKL: dll C:/Users/ALuo/Downloads/incubator-mxnet/build/mklml/mklml_win_2018.0.3.20180406/lib/mklml.dll
-- Intel(R) MKL: OpenMP dll C:/Users/ALuo/Downloads/incubator-mxnet/build/mklml/mklml_win_2018.0.3.20180406/lib/libiomp5md.dll
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- VTune profiling environment is unset
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_INTEL_LIBRARY MKLDNN_LIBRARY)
-- Could not find OpenBLAS include. Turning OpenBLAS_FOUND off
-- Could not find OpenBLAS lib. Turning OpenBLAS_FOUND off
CMake Error at cmake/Modules/FindOpenBLAS.cmake:82 (MESSAGE):
  Could not find OpenBLAS
Call Stack (most recent call first):
  3rdparty/mshadow/cmake/mshadow.cmake:26 (find_package)
  CMakeLists.txt:246 (include)

Edit:

Calling nvcc --version returns:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Wed_Apr_11_23:16:30_Central_Daylight_Time_2018
Cuda compilation tools, release 9.2, V9.2.88

[yajiedesign]

do you install cuda visual studio integration?

[aluo-x]

do you install cuda visual studio integration?

I did not install CUDA visual studio integration, since I did not install the entire visual studio 2017 package but only the build tools. I recall trying a build with build tools and CUDA with the visual studio integration, and the build still failed.

[yajiedesign]

@aluo-x

1. install cuda visual studio integration

2. install visual studio v14.11

3. use call "c:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvars64.bat" -vcvars_ver=14.11 chose v14.11

4. add -T cuda=9.2,host=x64 to cmake arg

[aluo-x]

@yajiedesign Thanks for the help. I managed to get the build working with VS 2017 Update 6, CUDA 9.2 and cuDNN 7.1.4, OpenCV 3.4.1, and MKL 2018 Update 3.

I noticed a few mistakes in the build instructions provided here.

Here are more detailed instructions provided here in case it might help others.

1. Download VS 2017 Update 6 Community from https://docs.microsoft.com/en-us/visualstudio/productinfo/installing-an-earlier-release-of-vs2017.
2. Install with options configured for C++ desktop development
3. Download CUDA 9.2.88 and CUDA 9.2.88.1, install with visual studio integration support enabled.
4. Install GPU drivers after CUDA installation, due to an existing bug breaking the Nvidia Control Panel if CUDA is installed after
5. Download OpenCV opencv-3.4.1 for Windows and extract
6. Download and install Intel MKL 2018 Update 3 and install
7. Run the following:
    "setx OpenCV_DIR "OPENCVDIR\build\x64\vc15\include"
    "setx MKL_ROOT "C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018\windows\mkl"
8. Add OPENCVDIR\build\x64\vc15\bin to path
9. Clone Mxnet and start the build:
    "git clone https://github.com/apache/incubator-mxnet.git --recursive"
    "cd incubator-mxnet"
    "mkdir build"
    "cmake -G "Visual Studio 15 Win64" .. -DUSE_CUDA=1 -DUSE_CUDNN=1 -DUSE_NVRTC=0 -DUSE_OPENCV=1 -DUSE_OPENMP=1 -DUSE_PROFILER=1 -DUSE_BLAS=mkl -DUSE_LAPACK=0 -DUSE_DIST_KVSTORE=0 -DCUDA_ARCH_NAME=6.1 -DUSE_MKLDNN=0 -DCMAKE_BUILD_TYPE=Release"
    "msbuild mxnet.sln"