Onednn: Intend to package mkl-dnn for Debian (and Ubuntu)

[cdluminate]

FYI: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=894411

It seems that the Apache-2 licensed mkl-dnn can be built and used without MKL, despite of a suboptimal performance. In this case we can make packages.

[vpirogov]

@CDLuminate, thanks for the update. Indeed the library built from the source is fully functional. We are also working on closing the performance gap between JIT GEMM implementation available in MKL-DNN and MKL.

Could you please elaborate on what issues you see with using libmklml in the package?

[cdluminate]

I'll deal with the packaging soon. I'll post issues discovered during the process here if any. And glad to hear that open sourced MKL-DNN is trying to catch up MKL in some aspects.

libmklml: Debian community is cautious at the licensing stuff. I took a look at its license file and found this. It feels like a BSD style license.

    * Redistributions of source code must retain the above copyright notice,
      this list of conditions and the following disclaimer.
    * Redistributions in binary form must reproduce the above copyright
      notice, this list of conditions and the following disclaimer in the
      documentation and/or other materials provided with the distribution.

But I didn't found source of mklml on github. Could you please point me to the source if available?
If the code of mklml is available, I'll be happy to package that. Besides, uploading redistributable pre-built binary is not preferred by the community.

MKL: Last time when I was trying to download MKL, registration was required. I assumed that the MKL binary is not redistributable. Is it still non-redistributable?

[cdluminate]

Well I found this https://software.intel.com/en-us/mkl/license-faq
Maybe a discussion with other community members is needed to decide whether to make a pkg for it.

[cdluminate]

Here is the discussion about MKL https://lists.debian.org/debian-science/2018/03/msg00070.html

[vpirogov]

Intel MKL is available under Intel Software Simplified License and allows redistribution. Downloading Intel MKL package from Intel Registration Center requires registration, however once downloaded there's no restrictions on redistribution. The package is also available without registration via yum and apt repositories. The source code for Intel MKL is not available.

To simplify enabling of deep learning applications we also provide a subset of Intel MKL functionality including BLAS and selected LAPACK and Vector Math functions in special distribution called Intel MKL small libraries or libmklml. This package is available for all supported platforms and allows redistribution. The source code for this package is not available.

This library, Intel MKL-DNN, includes the latest and greatest optimizations for deep learning functionality and all the source code is available on Github. There's an option to use SGEMM function from either Intel MKL or libmklml, which has better performance in some cases in comparison to open source implementation of this function available in Intel MKL-DNN.

[cdluminate]

@vpirogov Thank you for the elaborate explanations. I also noticed that the mkl core libraries are available via pip3 install mkl. And I have questions:

1. mklml is a subset of MKL. Does that mean I can make a mklml package by picking several specific libraries from the FULL version of MKL?

2. There are too many versions of MKL binary tarballs (e.g. apt/rpm/whl files provided by Intel). If we are about to package MKL, which tarball do you recommended to use?

3. Is there a doc describing the shared objects and the relations between the shared objects provided by MKL? High quality Debian packaging requires us to carefully split files into different packages. With that doc I can understand what are they, and how they depend on (linked against) each other.

The mkl-dnn packaging (WIP) is hosted here, and the link won't change.

[vpirogov]

There are a few levels of granularity available in Intel MKL when it comes to extracting a subset.

First you can choose specific libraries based on the usage model. The complete Intel MKL package includes several variants for each library including static and dynamic versions, IA32 and Intel64 variants, and variants necessary to accommodate for ABI differences between supported compilers. There are also separate libraries for MPI support. Description of all the libraries included into Linux package is available here.

Another option the library provides is building a shared object including a subset of the functionality. This is how we build libmklml.

All the Intel provided packages for Intel MKL are updated at the same time and provide the same functionality, with the exception of Conda package that does not include examples and Fortran functionality. So feel free to choose the channel that works best for you.

Carefully separating the library into packages might be tricky, I hope the link to documentation I provided will help. You can also examine structure of yum or apt packages to see how these are structured into components. Feel free to ping me if you need help understanding what individual libraries do and how to package these in the structure you need.

[cdluminate]

The mkl-dnn package is basically ready (built without MKL). However a test failure was encountered:
https://github.com/intel/mkl-dnn/issues/208

[cdluminate]

@vpirogov The information about MKL is helpful to packaging. I've created an empty repo here. And let's see if there will be more volunteers to help with this.

[cdluminate]

Some more questions about MKL-DNN for finalizing the package:

1. What architectures do you support? Currently debian has build machines of many architectures.
   I guess the architectures supported by you are intel64(amd64) and i?86 since mkl-dnn is created by Intel. However, hopefully maybe the default code branch (no SSE* or AVX* or alike) is portable to other architectures?

Here is a list of Debian's build machines. We hope that the uploaded packages work on as many architectures as possible.

amd64 (aka Intel64, x86_64)
arm64
armel
armhf
i386 (includes i486, i586, i686)
mips
mips64el
mipsel
ppc64el
s390x
alpha
hppa
hurd-i386
ia64
kfreebsd-amd64
kfreebsd-i386
m68k
powerpc
powerpcspe
ppc64
riscv64
sh4
sparc64
x32

[rsdubtso]

MKL-DNN has only been tested on intel64 machines, but I know that there are some forks that do work on non-x86 CPUs. The library has not been designed with non-x86 machines in mind, and the way it checks for the CPU features is not portable. So I would only package MKL-DNN for amd64.

[cdluminate]

Maybe this is the last question before upload:

~/p/m/m/examples ❯❯❯ g++ simple_net_int8.cpp -lmkldnn
~/p/m/m/examples ❯❯❯ ./a.out 
status: 5
message: could not create a convolution forward primitive descriptor

I see no special requirement of this code in CMakeLists.txt, is this a bug?

[cdluminate]

Same here http://debomatic-amd64.debian.net/distribution#experimental/mkl-dnn/0.13~20180406-ga5f6077-1/autopkgtest

The source is the latest master.

[cdluminate]

It seems that the issue above is about a non-informational error output https://github.com/intel/mkl-dnn/issues/214 when AVX512-BW is not available. This doesn't block packaging and I'll continue.

Let's keep this issue open until mkl-dnn entered Ubuntu devel, and I'll keep updating in this issue.

[cdluminate]

The package is already waiting for review: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=895729 . We have to wait for some time until it enters the archive.

[cdluminate]

@vpirogov Hi, I'm trying to split the ~1GB version of MKL into small packages. The first question is that, can cluster libraries work without any component from the interface layer, thread layer and computational layer? (If so we can provide an individual cluster meta package)

I have bunch of other questions. They'll be listed and post here later.

Here describes how I split them in Debian-specific format:
https://salsa.debian.org/science-team/intel-mkl/blob/master/debian/control (Work in progress, anything may be changed at anytime).

[emfomenk]

Hi @CDLuminate,

can cluster libraries work without any component from the interface layer, thread layer and computational layer?

No. Intel MKL has 3 mandatory layers:

• interface (`lp64, ilp64, gnu fortran lp64, gnu fortran ilp64)
• threading (intel threading, gnu threading, sequential, intel tbb, pgi threading)
• core

Whenever you want to work with Intel MKL you have to include all these layers (one library per layer depending on the choice).
If you want to use cluster components in addition to the previous layers you have to link with:

• blacs (intelmpi, openmpi, ...)
• [optional] scalapack (lp64, ilp64)
• [optional] cdft

BLACS interface is mandatory, because it contains all the MPI bindings.

[cdluminate]

@vpirogov Hi, a license problem:

e.g. opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/include/fftw/fftw.h:

!   This program is distributed with permission
!

Under what license are these files distributed? Are they distributed under BSD-2-Clause like the nearby header files or under ISSL?

[cdluminate]

I've almost finished the packaging. Only several small issues remain to be fixed.

[cdluminate]

I manually mirrored the packaging repo to here

[vpirogov]

@CDLuminate, Intel MKL, including the header you are referring to, is distributed under Intel Simplified Software License. This license allows redistribution.

[cdluminate]

@vpirogov Thanks for the confirmation. The package is now waiting for review.

[rsdubtso]

@cdluminate Any news?

[cdluminate]

@rsdubtso MKL is still waiting to be manually checked by Debian's ftp masters. See https://ftp-master.debian.org/new/intel-mkl_2018.3.222-1.html

mkl-dnn is still waiting for a sponsor to be uploaded to ftp-master. Shall I hurry up a bit on it? I'm recently updating the Julia package.

[kolbusa]

I'm not trying to expedite this or anything. I was just wondering if I'm reading the packaging status correctly. Please take your time. And thanks for this!

[cdluminate]

@kolbusa OK. The time that MKL has to wait in the "NEW QUEUE" is not what I can decide and what I can do now is just wait. I can make the progress on mkl-dnn a bit faster but now it's blocked by #284 and #283 .

[cdluminate]

FYI: MKL has landed on Debian unstable.

• [x] MKL
• [x] mkl-dnn

[cdluminate]

One more question: Can AMD cpus gain any performance boost from intel's mkl-dnn library?

I'm trying to package tensorflow for Debian, I need the answer of the above question before having the tensorflow package linked against mkl-dnn by default. I've never had any AMD cpus in my life so I cannot figure it out by myself.

[vpirogov]

Yes, AMD processors are x86 compatible and with a few exceptions use the same Instruction Set Architecture (ISA) as Intel processors.

[eLvErDe]

Hi,

http://packages.le-vert.net/machinelearning/debian/pool-stretch/

I already made package for mkl dnn and tensorflow, if you're interrested in.

Adam.

[cdluminate]

@eLvErDe Nice work! Debian's mkl-dnn packaging is available here https://salsa.debian.org/science-team/mkl-dnn . After reading your packaging scripts I realized there is a giant gap between our scripts... As much as I'd love to compile packages with SIMD instruction sets enabled, or linked against MKL, such resulting packages won't enter Debian official archive...

[eLvErDe]

Exactly. Tensorflow is hopeless, bazel, Cuda... Have you seen the insane shit I have to do to populate a folder full of symlink to make Devian's CUDA look like it's upstream tarball ?
Anyway, I'll probably rebuild your mkn and mkl dnn package for my own use later thanks for handling this :-)

[eLvErDe]

BTW, there is at least boinc I think in the archive coming as to separate package, like boinc and boinc-contrib, one of them is in contrib and is linked to CUDA. You may use the same workaround to provide an mkl-enabled version of mkl-dnn....

[cdluminate]

Exactly. Tensorflow is hopeless, bazel, Cuda...

I wrote a very hacky and experimental build system for tensorflow https://salsa.debian.org/science-team/tensorflow, which requires only python3 and ninja. The build system reads bazel's query results and decide how to compile specified targets. Well, to make this packaging reasonably useful enough, more work is required...

Have you seen the insane shit I have to do to populate a folder full of symlink to make Devian's CUDA look like it's upstream tarball ?

I'm sorry to hear that, since I'm also one of debian's cuda toolkit package maintainer. We have to mangle the installation path to make things compliant to the FHS standard and Debian policy...

Anyway, I'll probably rebuild your mkn and mkl dnn package for my own use later thanks for handling this :-)

Feel free to send me patches, if some portion of your packaging work can be merged to upstream packaging to reduce your maintenance burden a bit.

@eLvErDe

[cdluminate]

BTW, there is at least boinc I think in the archive coming as to separate package, like boinc and boinc-contrib, one of them is in contrib and is linked to CUDA. You may use the same workaround to provide an mkl-enabled version of mkl-dnn....

I'm also maintainer of Debian's caffe package .... there is already src:caffe and src:caffe-contrib, where caffe-contrib is linked against CUDA (but not cuDNN because cuDNN is still not available in archive). Yes, the same logic applies to Debian's mkl-dnn package, but I just don't want to copy everything twice and increase my maintenance burden. By the way, src:intel-mkl is also maintained by me... https://tracker.debian.org/pkg/intel-mkl ... Every package that links against MKL or CUDA would simply double my burden... Sigh...

@eLvErDe

[eLvErDe]

Regarding CUDA, it's definitely not your fault but bazel's stupid behavior....
I saw the bazel to ninja convert, insane and brillant at the time ! You may be interested in my Python script wrapping upstream configure script to generate different set of bazel rules.

Anyway, would you be interested in having a chat regarding this somewhere ?

[cdluminate]

@eLvErDe If you don't mind using public mailing list, https://lists.debian.org/debian-science/ is a better place for the discussion. I subscribed the list, but feel free to CC me if you want.

[rsdubtso]

Hi @cdluminate and @eLvErDe ! Thanks for doing the packaging work! Feel free to CC me on the discussions related to packaging MKL and MKL-DNN as well. If there is anything we can do to make packaging -- please let us know.

[cdluminate]

Finally! Debian's ftp team has accepted mkl-dnn into experimental. I'm closing this issue now since intel-mkl and mkl-dnn are both in the archive. I'll soon upload it to Debian unstable, and mkl-dnn will eventually enter testing and stable.

https://tracker.debian.org/pkg/mkl-dnn
https://tracker.debian.org/intel-mkl

[vpirogov]

Thank you for the great work, @cdluminate!

[cdluminate]

mkl-dnn 0.17~rc has been automatically pulled into Ubuntu's development branch. Package status can be tracked here:
https://launchpad.net/ubuntu/+source/mkl-dnn

[RoyiAvital]

@cdluminate , Pay attention that as far as I know Intel MKL does discriminate non Genuine Intel CPU's.

@vpirogov , you wrote:

Yes, AMD processors are x86 compatible and with a few exceptions use the same Instruction Set Architecture (ISA) as Intel processors.

Does it mean MKL-DNN checks only for ISA features and not the manufactur?
Is there a change in MKL itself as you now distribute it for Linux distributions? As MKL used to use CPU dispatcher which looks for Genuine Intel.

Thank You.

[vpirogov]

@RoyiAvital, Intel MKL-DNN dispatches the code based on ISA features only. Here's the relevant piece of code.

[RoyiAvital]

@vpirogov , What happens if one links it to MKL?
Does MKL use generic code path on AMD?

[vpirogov]

@RoyiAvital, since v1.0 Intel MKL-DNN has fully optimized JIT GEMM and does not support linking with Intel MKL anymore. So starting from Intel MKL-DNN v1.0 the question is moot.

Intel MKL may or may not optimize to the same degree for non-Intel microprocessors. For more complete information about optimizations for non-Intel processors see Optimization Notice.

[RoyiAvital]

Intel MKL may or may not optimize to the same degree for non-Intel microprocessors. For more complete information about optimizations for non-Intel processors see Optimization Notice.

The problem is the optimization notice says we should look into the product documentation yet no where in MKL documentation it says when it will chose code path based on CPU feature or when based on manufacture.

I really hope the MKL team will embrace your policy and set the code path based only on the CPU features and not the manufacture.

Thank You.

[vpirogov]

@RoyiAvital, thank you for the input. I will bring it to the attention of Intel MKL team.