Dlib: MSVC 2015 supports C++ 11 now

[e-fominov]

Hello, yesterday I have downloaded latest MSVC 2015 Update 3.
Compiler version is: 19.00.24215.1

And I have successfully compiled all DNN examples (CPU mode only) and also test_for_cpp11 compiles well too

To build examples I put explicit flag:
cmake .. -DCOMPILER_CAN_DO_CPP_11=ON

And imagenet exmples - both example and trainer are not compiling well - compiler still goes into infinite loop. But previous versions were unable to compile dnn_introduction2_ex. Now it compiles as-is

I have tried to find the reason why does it not compile and looks like DNN structure is too-templated for MSVC.
I made small changes and it compiled well:

template <typename SUBNET> using level1 = ares<512,ares<512,ares_down<512,SUBNET>>>;
template <typename SUBNET> using level2 = ares<256,ares<256,ares<256,ares<256,ares<256,ares_down<256,SUBNET>>>>>>;
template <typename SUBNET> using level3 = ares<128,ares<128,ares<128,ares_down<128,SUBNET>>>>;
template <typename SUBNET> using level4 = ares<64,ares<64,ares<64,SUBNET>>>;
using anet_type = loss_multiclass_log<fc<1000,avg_pool_everything<
                            level1<
                            level2<
                            level3<
                            level4<
                            max_pool<3,3,2,2,relu<affine<con<64,7,7,2,2,
                            input_rgb_image_sized<227>
                            >>>>>>>>>>>;

So I think, now we can use MSVC 2015 with dlib. Waiting only for CUDA updates

[davisking]

I know, that's the version of MSVC I've been using. I told the MSVC developers about the infinite compiler loop and supposedly they are working on fixing it.

I'm not super sure I would call this totally supporting C++11 now though. But maybe you are right that it's good enough if we adjust the examples to avoid the bug in MSVC.

[e-fominov]

I can make PR. do you agree with the method of how the sample was changed?
May be some renaming?

[davisking]

The change looks right. I would test the imagenet example on some images and make sure its outputs are the same before and after the change though (using the pretrained model).

It would be nice if a single template alias could be created that was something like block. You can do that using the repeat<> layer type. That's what I would do and is probably even easier to read than what's in there now. Although I don't know if that would upset MSVC.

[grisevg]

What CUDA updates are you waiting for? Isn't CUDA 8 working with VS2015?

[e-fominov]

Cuda 8.0.27 does not support VS2015 update >= 2 and Dlib need VS 2015 update >= 3:
c:\program files\nvidia gpu computing toolkit\cuda\v8.0\include\host_config.h(137): fatal error C1189: #error: Microsoft Visual Studio 2015 Update >= 2 is not supported yet! [C:\work\dlib\dlib\cmake_utils\test_for_cuda\build\cuda_test.vcxproj]

[grisevg]

I guess it's time to install linux...

[davisking]

lol, pretty much.

[jeandebleau]

Hi; it seems like cuda 8 has been released and supports the update 3. https://devblogs.nvidia.com/parallelforall/cuda-8-features-revealed/

I will test it as soon as possible.
And again, thanks to everybody contributing to this awesome library !

[davisking]

Sweet, thanks :)

[e-fominov]

Tested with latest cuda 8.0.44 and cudnn 5.1 on windows 7 x64, MSVC compiler version 19.0.24215.1:

C:\work\dlib\b15>cmake .. -G"Visual Studio 14 2015 Win64" -DCOMPILER_CAN_DO_CPP_11=ON
-- The C compiler identification is MSVC 19.0.24215.1
-- The CXX compiler identification is MSVC 19.0.24215.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Searching for BLAS and LAPACK
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Found CUDA: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0 (found suitable version "8.0", minimum required is "7.5")
-- Looking for cuDNN install...
C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/lib/x64/cublas.libC:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/lib/x64
C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/lib/x64/cudnn.lib
-- Building a CUDA test project to see if your compiler is compatible with CUDA...
-- Checking if you have the right version of cuDNN installed.
C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/lib/x64/cublas.libC:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/lib/x64
C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/lib/x64/cudnn.lib
-- Found cuDNN: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/lib/x64/cudnn.lib
-- Configuring done
-- Generating done
-- Build files have been written to: C:/work/dlib/b15

When running dnn samples I see this error message:
Error while calling cudaGetDevice(&dev) in file C:/work/dlib/dlib/dnn/cuda_dlib.cu:26. code: 38, reason: no CUDA-capable device is detected

This means that sample compiled with CUDNN support and when I will put it on machine with good GPU - it will work

Can anybody make real testing on Windows with good GPU?

[davisking]

Cool. I'll give it a try later this evening.

[grisevg]

@e-fominov
I've got GTX960M with Optimus which usually causes lots of issues, but official CUDA samples compile and run just fine.

But I must have done something wrong with dlib or CUDA setup. I've got the latest Visual Studio Professional 2015 Update 3 and yet when I run CMake (latest 3.6.2) it sais that:

...
-- Performing Test COMPILER_SUPPORTS_CXX0X - Failed
-- C++11 activated (compiler doesn't have full C++11 support).
...
-- *** Dlib CUDA support requires C++11 but your compiler doesn't support it. ***

and skips dnn examples.

I've had to force set COMPILER_CAN_DO_CPP_11 or otherwise dlib would disable all dnn code.
And then I had to modify find_cudnn.txt to look for the cuDNN library in lib/x64/ in addition to just lib.

Then I've compiled and ran dnn_introduction_ex and it worked just fine. It's using GPU and training one epoch takes about 20 seconds. 😄 🎉 💃

[e-fominov]

Yes, COMPILER_CAN_DO_CPP_11 flag is required to enable VS2015 compiling dnn examples.
When I was compiling dlib, I put CUDNN files into the same folders where the main CUDA libs were placed and I didn't need to make any modifications to find_cudnn.txt. I think, that for Windows CUDNN should be placed into Cuda folder. But we can make Cmake find in CMAKE_PREFIX_PATH too

I think we should change CMakeFiles and turn back test_for_cpp11 for MSVC2015

[grisevg]

@e-fominov yeah I also have the cuDNN library in the same place where CUDA ones are: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib\x64 but find_cudnn.txt couldn't find it until I added lib/x64 to PATH_SUFFIXES.

Maybe you have environmental variable CUDNN_LIBRARY_DIR set? I don't.

Also, I don't think there's a way in CMake to tell if Update 3 was installed or no?

[davisking]

@grisevg CMake should find cuDNN as is. I suspect you didn't create a 64bit project? That's needed for CMake to find cuDNN (since it's 64bit). I also tested this and it works as expected when I placed cuDNN into the same folder you did.

@e-fominov I just tried to compile the examples in visual studio and it's still going into an infinite loop on dnn_introduction2_ex.cpp. I have MSVC update 3 installed. Although maybe there is more than one version of "update 3"? This is what my about screen looks like:

[davisking]

On the plus side dnn_introduction_ex worked :)

[jeandebleau]

I have tested the master branch with visual studio 2015 update 3, cuda 8 and cudnn 5.1. I also used the modifications suggested by e-fominov.

All the examples are compiling except the dnn_imagenet_train_ex which is running the compiler out of memory. I have tried to separate the file into two sets containing the training network and the testing network. Unfortunately the network of the imagenet containing the batch normalisation layer creates the error C10002: compiler is out of heap space in pass 2. Two much templates...

At this point I cannot train the defined network on the imagenet dataset but I can use the pretrained network.

Finally dnn_imagenet_ex is working and the results look ok:

Thanks !

[jeandebleau]

@davisking Yes there are multiple "update 3" :(

You should try to download manually the version from the 09/14/2016 which is the version 14.0.25431.01 ; which is the one I am using: https://msdn.microsoft.com/en-us/library/mt752379.aspx

[e-fominov]

I have successfully compiled all DNN examples with VS 2015 UP3 and Cuda 8.1. My code is in #269
Compiler version is 19.00.24215.1 - this version is printed by CMAKE or when simply running CL from command line. @davisking , Visual Studio/about says its version is: 14.0.25431.01 Update 1 - it is different from yours

And I got the same problem as @jeandebleau - error C1002 - not enough heap. To solve it I used x64 compiler from amd64 folder, compilation process take 25 minutes and more than 4 GB RAM for dnn_imagenet_train_ex example

[davisking]

I set CMAKE_CXX_COMPILER to C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/amd64/cl.exe and still Visual Studio runs out of RAM. So I've told cmake to avoid compiling that one example when someone uses Visual Studio. I've also rolled in @e-fominov's changes to the examples that avoid problems with visual studio as well as a few other minor cmake things to make compilation smoother with visual studio.

[davisking]

The changes are in github now. Maybe when the next version of visual studio is out we can enable the imagenet training example :/