Incubator-mxnet: Default build with CMake not using MKL

[larroy]

With make MKL is installed automatically, but with CMake in linux not.

piotr@ip-172-31-30-72:0:~/mxnet (master)$ cp cmake/cmake_options.yml .
piotr@ip-172-31-30-72:0:~/mxnet (master)$ grep MKL cmake_options.yml
USE_MKL_IF_AVAILABLE: "ON" # Use MKL if found                                                                                                                                                                                                USE_MKLML_MKL: "ON" # Use MKLDNN variant of MKL (if MKL found) IF USE_MKL_IF_AVAILABLE AND (NOT APPLE)                                                                                                                                       USE_MKLDNN: "ON" # Use MKLDNN variant of MKL (if MKL found) IF USE_MKL_IF_AVAILABLE AND (NOT APPLE)                                                                                                                                          piotr@ip-172-31-30-72:0:~/mxnet (master)$ ./dev_menu.py menu 1 | tee build.log
INFO:root:CMake / Ninja build in directory /home/piotr/mxnet/build
INFO:root:Executing: cmake
-DUSE_CUDA=OFF
-DUSE_OLDCMAKECUDA=OFF
-DUSE_NCCL=OFF
-DUSE_OPENCV=ON
-DUSE_OPENMP=ON
-DUSE_CUDNN=ON
-DUSE_SSE=ON
-DUSE_F16C=ON
-DUSE_LAPACK=ON
-DUSE_MKL_IF_AVAILABLE=ON
-DUSE_MKLML_MKL=ON
-DUSE_MKLDNN=ON
-DUSE_OPERATOR_TUNING=ON
-DUSE_GPERFTOOLS=ON
-DUSE_JEMALLOC=ON
-DUSE_PROFILER=ON
-DUSE_DIST_KVSTORE=OFF
-DUSE_PLUGINS_WARPCTC=OFF
-DUSE_PLUGIN_CAFFE=OFF
-DUSE_CPP_PACKAGE=OFF
-DUSE_MXNET_LIB_NAMING=ON
-DUSE_GPROF=OFF
-DUSE_CXX14_IF_AVAILABLE=OFF
-DUSE_VTUNE=OFF
-DENABLE_CUDA_RTC=ON
-DBUILD_CPP_EXAMPLES=ON
-DINSTALL_EXAMPLES=OFF
-DUSE_SIGNAL_HANDLER=ON
-DUSE_TENSORRT=OFF
-DUSE_ASAN=OFF
-DENABLE_TESTCOVERAGE=OFF
-DCMAKE_BUILD_TYPE=Debug
-DCMAKE_CUDA_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-GNinja
..
-- CMAKE_CROSSCOMPILING FALSE
-- CMAKE_HOST_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_NAME Linux
-- Determining F16C support
-- Downloading MKLML...
CMake Error at cmake/DownloadMKLML.cmake:66 (file):
  file DOWNLOAD HASH mismatch

    for file: [/home/piotr/mxnet/build/mklml/mklml_lnx_2019.0.3.20190220.tgz]
      expected hash: [76354b74325cd293aba593d7cbe36b3f]
        actual hash: [d41d8cd98f00b204e9800998ecf8427e]
             status: [1;"Unsupported protocol"]

Call Stack (most recent call first):
  CMakeLists.txt:242 (include)




gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
-- Setting MKLROOT path to /home/piotr/mxnet/build/mklml/mklml_lnx_2019.0.3.20190220
-- This is a product build
-- Detecting Intel(R) MKL: trying mklml_intel
-- Detecting Intel(R) MKL: trying mklml_gnu
-- Detecting Intel(R) MKL: trying mklml
-- Detecting Intel(R) MKL: trying mkl_rt
CMake Warning at 3rdparty/mkldnn/cmake/MKL.cmake:267 (message):
  Intel(R) MKL not found.  Some performance features may not be available.
  Please run scripts/prepare_mkl.sh to download a minimal set of libraries or
  get a full version from https://software.intel.com/en-us/intel-mkl
Call Stack (most recent call first):
  3rdparty/mkldnn/cmake/OpenMP.cmake:25 (include)
  3rdparty/mkldnn/CMakeLists.txt:76 (include)


-- Found OpenMP_C: -fopenmp
-- Found OpenMP_CXX: -fopenmp
-- OpenMP lib: provided by compiler
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- VTune profiling environment is unset
CMake Warning (dev) at cmake/ChooseBlas.cmake:23 (find_package):
  Policy CMP0074 is not set: find_package uses <PackageName>_ROOT variables.
  Run "cmake --help-policy CMP0074" for policy details.  Use the cmake_policy
  command to set the policy and suppress this warning.

  CMake variable MKL_ROOT is set to:

    MKL_ROOT-NOTFOUND

  For compatibility, CMake is ignoring the variable.
Call Stack (most recent call first):
  CMakeLists.txt:298 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_GNU_LIBRARY MKLDNN_LIBRARY)
-- Found OpenBLAS libraries: /usr/lib/x86_64-linux-gnu/libopenblas.so
-- Found OpenBLAS include: /usr/include/x86_64-linux-gnu
-- Could NOT find MKL (missing: MKL_INCLUDE_DIR MKLML_GNU_LIBRARY MKLDNN_LIBRARY)
-- Found OpenBLAS libraries: /usr/lib/x86_64-linux-gnu/libopenblas.so
-- Found OpenBLAS include: /usr/include/x86_64-linux-gnu
-- Could NOT find Gperftools (missing: GPERFTOOLS_LIBRARIES GPERFTOOLS_INCLUDE_DIR)
-- Using JEMalloc malloc
--  OpenCV_LIBS=opencv_core;opencv_highgui;opencv_imgproc;opencv_imgcodecs
-- OpenCV 3.2.0 found (/usr/share/OpenCV)
-- LIBOMP: Operating System     -- Linux
-- LIBOMP: Target Architecture  -- x86_64
-- LIBOMP: Build Type           -- Debug
-- LIBOMP: OpenMP Version       -- 50
-- LIBOMP: Library Kind         -- SHARED
-- LIBOMP: Library Type         -- normal
-- LIBOMP: Fortran Modules      -- FALSE
-- LIBOMP: Build                -- 20140926
-- LIBOMP: Use Stats-gathering  -- FALSE
-- LIBOMP: Use Debugger-support -- FALSE
-- LIBOMP: Use ITT notify       -- TRUE
-- LIBOMP: Use OMPT-support     -- FALSE
-- LIBOMP: Use Adaptive locks   -- TRUE
-- LIBOMP: Use quad precision   -- TRUE
-- LIBOMP: Use TSAN-support     -- FALSE
-- LIBOMP: Use Hwloc library    -- FALSE
-- LIBOMP: Cannot find llvm-lit.

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Build

[pengzhao-intel]

thank to report this issue @larroy

@Neozhangjianyu @yinghu5 is working on the CMake for MKL

[larroy]

@mxnet-label-bot add [CMake, Build, MKL]

[NeoZhangJianyu]

@larroy I'm fixing this issue now.
There are several problems for this issue:

1. $MKLROOT is set by MKL installation, but this cmake expect $MKL_ROOT.
2. If MKLDNN is used, it will use OpenBLAS, instead of MKL as BLAS.
3. Wrong dependence check in MKLDNN and MKL. It will make the MKL check is fault.

I have fixed it in linux locally. But the solution need to be reviewed.
And we need to fix it in windows build too. (The cmakefile support linux, windows and mac in same time)
So it will take more days to be verified and committed.

[pengzhao-intel]

cc @TaoLv

[larroy]

Thanks for the great response guys, you rock!

[sl1pkn07]

@NeoZhangJianyu can you also fix this?

~~~
CMake Warning (dev) at 3rdparty/mkldnn/cmake/utils.cmake:120 (target_link_libraries):
Policy CMP0023 is not set: Plain and keyword target_link_libraries
signatures cannot be mixed. Run "cmake --help-policy CMP0023" for policy
details. Use the cmake_policy command to set the policy and suppress this
warning.

The plain signature for target_link_libraries has already been used with
the target "mkldnn". All uses of target_link_libraries with a target
should be either all-keyword or all-plain.

The uses of the plain signature are here:

• 3rdparty/mkldnn/cmake/utils.cmake:111 (target_link_libraries)

Call Stack (most recent call first):
3rdparty/mkldnn/src/CMakeLists.txt:108 (target_link_libraries_install)
This warning is for project developers. Use -Wno-dev to suppress it.

~~~

with cmake 3.14.2

greetings

[NeoZhangJianyu]

@sl1pkn07
It's a warning, you could ignore it.

Or fix it manually:

Edit 3rdparty/mkldnn/cmake/utils.cmake:111:

From

function(target_link_libraries_build target list)
    # Foreach is required for compatibility with 2.8.11 ways
    foreach(lib ${list})
            target_link_libraries(${target}
            "$<BUILD_INTERFACE:${lib}>")
    endforeach(lib)
endfunction()

to

function(target_link_libraries_build target list)
    # Foreach is required for compatibility with 2.8.11 ways
    foreach(lib ${list})
            target_link_libraries(${target} PUBLIC
            "$<BUILD_INTERFACE:${lib}>")
    endforeach(lib)
endfunction()

[sl1pkn07]

yep. but should be "fixed"/silenced in upstream, or i'am wrong?

greetings

[NeoZhangJianyu]

Yes. It's an issue of mkldnn and should be fixed.
I have created an issue to mkldnn repo.
https://github.com/intel/mkl-dnn/issues/457

[sl1pkn07]

ok. my issue is fixed via https://github.com/intel/mkl-dnn/commit/e35da48e4a00fe1ae2fa079f877a78a98df6bf6a, now wait to bump the version in mxnet .gitmodules

greetings

[TaoLv]

@sl1pkn07 Thank you for your patience. We're working on a new release of MKL-DNN. Will update MXNet submodule once the release comes out.

[NeoZhangJianyu]

@larroy This issue has been fixed totally by https://github.com/apache/incubator-mxnet/pull/14877.
You could try in latest master branch.

[larroy]

Works nicely now, tested with MKL, no mkl and MKLDNN... thanks guys.

[gasabr]

i'm trying to build mxnet version 1.5.0 from sources and still receive the same error (in 1.5.1 as well) could you please help me with this? Below goes context of my dockerfile, so the error should be trivial to reproduce

FROM ubuntu:18.04 as build
LABEL description="build images for the extractor server"

# next 2 lines help avoid configuring tzinfo in docker
ENV TZ=Europe/Moscow
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezon
RUN apt update \
    && apt install -y git g++ build-essential automake libtool libssl-dev \
       libcpprest-dev wget \
       libopenblas-dev liblapack-dev libopencv-dev graphviz \
       libjemalloc-dev libatlas-base-dev \
       python3.8 \
    && rm -rf /var/lib/apt/lists/*

# installing cmake, because apt installs 3.10, when 3.13 is needed
RUN git clone --depth 1 --branch v3.17.0 https://github.com/Kitware/CMake/ \
    && cd CMake \
    && ./bootstrap -- -DCMAKE_USE_OPENSSL=OFF && make && make install

ENV OPENCV_VERSION=4.1.1
# Building and installing `OpenCV`
RUN wget https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.tar.gz \
    -O /usr/src/opencv-${OPENCV_VERSION}.tar.gz \
    && tar -xzf /usr/src/opencv-${OPENCV_VERSION}.tar.gz -C /usr/src \
    && cd /usr/src/opencv-${OPENCV_VERSION}/ && mkdir -p build && cd build \
    && cmake -DBUILD_SHARED_LIBS=OFF -DOPENCV_FORCE_3RDPARTY_BUILD=ON \
    -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DWITH_1394=OFF -DWITH_FFMPEG=OFF \
    -DWITH_GTK=OFF -DWITH_GSTREAMER=OFF .. && make install -j $(nproc) \
    && rm /usr/src/opencv-${OPENCV_VERSION}.tar.gz

# install mkl libs
RUN cd /tmp && \
    # now get the key:
    wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB && \
    # now install that key
    ls && \
    apt-key add ./GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB && \
    # now remove the public key file exit the root shell
    rm GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB && \
    wget https://apt.repos.intel.com/setup/intelproducts.list -O /etc/apt/sources.list.d/intelproducts.list &&\
    apt-get update && \
    apt-get install -y intel-mkl-2020.0-088

WORKDIR /usr/src
RUN git clone --depth 1 --branch 1.5.0 --recursive https://github.com/apache/incubator-mxnet
RUN apt install -y python
RUN cd /usr/src/incubator-mxnet/ && \
    mkdir build && cd build && \
    cmake \
        -DUSE_INTEL_PATH=/opt/intel \
        -DUSE_CUDA=0 \
        -DUSE_CPP_PACKAGE=1 \
        -DUSE_CUDNN=0 \
        -DUSE_MKLDNN=1 \
        -DUSE_MKL_IF_AVAILABLE=0 \
        -DUSE_MKLML=0 \
        -DUSE_OPENCV=1 \
        -DMXNET_PREDICT_ONLY=1 \
        -DCMAKE_BUILD_TYPE=Release \
        -DUSE_JEMALLOC=ON \
        -DUSE_LAPACK=OFF \
        -DBUILD_CPP_EXAMPLES=OFF \
        .. && \
    make install -j$(nproc)