Caffe: is support Visual Studio 2015, CUDA 9.x, Python 3.5/3.6/3.7?

[cuixing158]

Dose this caffe support Visual Studio 2015/2017, CUDA 9.x, Python 3.5/3.6/3.7?

Because now with the upgrade of hardware devices, many have not been updated, I hope the official can solve such problems, thank you！

[Luminisc]

+1, I have VS 2017, CUDA 9 and Python 3.7, and I can't build project because there are only settings for VS2015 + Python 3.5

[willyd]

I will merge a PR (in a few days) that improves the windows CMake build and supports building with vcpkg and conda. Once few people validate that those packages are working I will merge the changes to the windows branch.

Installation should be as simples as:

conda create -n caffe
conda config --add channels anaconda
conda install caffe -c willyd

I will post back when this is ready to test.

I currently have packages for win64 caffe cpu, CUDA 9.0, python 3.5, 3.6, 3.7.

[willyd]

I have published conda packages. Please install with these commands:

REM Create a new conda environment to install Caffe
REM You can use python 3.5, 3.6 or 3.7
conda create -n caffe python=3.7
REM activate the environment
REM Depending on your conda version you may have to use activate caffe
conda activate caffe
REM add the anaconda channel to resolve all dependencies
conda config --add channels anaconda
REM Install caffe with cuda support or install caffe-cpu if you do not want CUDA support
conda install caffe -c willyd

Currently only CUDA 9.0 is built but I will merge the files to build the packages once I have confirmation that the packages work.

[Franzisdrak]

conda config --add channels ananconda

Careful when copying there is a small typo at channels "anaNconda"

[willyd]

Thanks @Franzisdrak I edited the comment accordingly. Have you tried the conda packages? Did they work for you?

[Franzisdrak]

@willyd Yeah i am trying. But i am running into one issue after another. Currently:

INFO: ============================================================
INFO: Summary:
INFO: ============================================================
INFO: MSVC_VERSION               = 14
INFO: WITH_NINJA                 = 1
INFO: CMAKE_GENERATOR            = "Ninja"
INFO: CPU_ONLY                   = 0
INFO: CUDA_ARCH_NAME             = Auto
INFO: CMAKE_CONFIG               = Release
INFO: USE_NCCL                   = 0
INFO: CMAKE_BUILD_SHARED_LIBS    = 0
INFO: PYTHON_VERSION             = 3
INFO: BUILD_PYTHON               = 1
INFO: BUILD_PYTHON_LAYER         = 1
INFO: BUILD_MATLAB               = 1
INFO: PYTHON_EXE                 = "python"
INFO: RUN_TESTS                  = 0
INFO: RUN_LINT                   = 0
INFO: RUN_INSTALL                = 0
INFO: ============================================================

Please make sure either Visual Studio or C++ Build SKU is installed.
-- The C compiler identification is MSVC 19.0.24225.1
-- The CXX compiler identification is MSVC 19.0.24225.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/amd64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/amd64/cl.exe -- broken
CMake Error at C:/Users/siege/Anaconda3/envs/caffe/Library/share/cmake-3.12/Modules/CMakeTestCCompiler.cmake:52 (message):
  The C compiler

    "C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/amd64/cl.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: C:/Users/siege/Desktop/Novelsense/caffe-master/build/CMakeFiles/CMakeTmp

    Run Build Command:"C:/Users/siege/Anaconda3/envs/caffe/Library/bin/ninja.exe" "cmTC_46157"
    [1/2] Building C object CMakeFiles\cmTC_46157.dir\testCCompiler.c.obj
    [2/2] Linking C executable cmTC_46157.exe
    FAILED: cmTC_46157.exe
    cmd.exe /C "cd . && C:\Users\siege\Anaconda3\envs\caffe\Library\bin\cmake.exe -E vs_link_exe --intdir=CMakeFiles\cmTC_46157.dir --manifests  -- C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\link.exe /nologo CMakeFiles\cmTC_46157.dir\testCCompiler.c.obj  /out:cmTC_46157.exe /implib:cmTC_46157.lib /pdb:cmTC_46157.pdb /version:0.0  /machine:x64  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."
    RC Pass 1: command "rc /foCMakeFiles\cmTC_46157.dir/manifest.res CMakeFiles\cmTC_46157.dir/manifest.rc" failed (exit code 0) with the following output:
    Das System kann die angegebene Datei nicht finden
    ninja: build stopped: subcommand failed.




  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:13 (project)


-- Configuring incomplete, errors occurred!

I am not too experienced with all the Build Tools, as i am a dumb Windows user 😄

[gfursin]

Hi @Franzisdrak .

You can try another complementary approach to build Caffe on Windows using CK workflow framework. It attempts to automatically detect all installed software dependencies (Microsoft compilers, python frameworks, cmake, etc), install missing ones, and then prepare and run the build script (https://github.com/dividiti/ck-caffe).

Please see my notes how to do it here: https://github.com/BVLC/caffe/issues/5654#issuecomment-372123226 .

I just checked it with VS 2017, CUDA 9.2 and Python 3.6, and it worked.

Hope it can be of any help - it's a part of our on-going community effort to develop a portable workflow framework which can help (novice) end-users to automate detection and installation of all missing code and data dependencies across different platforms. If it's of interest, we plan to discuss it during the open panel at the Supercomputing'18 in two weeks: http://rescue-hpc.org .

CC @psyhtest & @ens-lg4 .

[willyd]

@Franzisdrak I meant the conda packages. Once installed you don't need to build anything. After using the above conda install caffe -c willyd and all previous commands you should be able to do (in a python interpreter session):

> import caffe
> # call caffe.Net to create a Network and perform inference
> # call caffe.SGDSolver to create a solver and perform training

Of course this won't help you much if you want to avoid python and use C++ only.

[willyd]

@Franzisdrak I know some people have had issues when using Windows in chinese languages. Could you try to switch your OS to the english language and see if the error remains?

[Franzisdrak]

@willyd Ok i will try this path tomorrow. Maybe even this evening. I decided to clean my whole system. I even broke my VS2017 installation during trying to fix it... This might have been the root cause.

Following error occurs after running your mentioned cmds:

>>>import caffe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\siege\Anaconda3\envs\caffe\lib\site-packages\caffe\__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
  File "C:\Users\siege\Anaconda3\envs\caffe\lib\site-packages\caffe\pycaffe.py", line 13, in <module>
    from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: DLL load failed: Das angegebene Modul wurde nicht gefunden.

Edit:
Ok so i need to adjust my Path variables according to https://github.com/BVLC/caffe/issues/5053? But i won't have time to do this today.

[Franzisdrak]

@willyd
I have troubles understanding what i actually need to do. As the simple installation in Anaconda doesn't seem to work for me.

As far as i understand it, i need to:

1. Install the python packages in anaconda

conda create -n caffe python=3.7
conda activate caffe
conda config --add channels anaconda
conda install caffe -c willyd

This works, but throws the mentioned error one post above when importing caffe. Therefore it seems i need to build the binaries myself. ->

1. Build the C++ Binaries from the windows caffe tree.
   I am following these instructions:
   https://github.com/BVLC/caffe/tree/windows#using-the-python-interface
   But i get this error:

INFO: ============================================================
INFO: Summary:
INFO: ============================================================
INFO: MSVC_VERSION               = 15 (Changed location in the build_win.cmd)
INFO: WITH_NINJA                 = 1
INFO: CMAKE_GENERATOR            = "Ninja"
INFO: CPU_ONLY                   = 0
INFO: CUDA_ARCH_NAME             = Auto
INFO: CMAKE_CONFIG               = Release
INFO: USE_NCCL                   = 0
INFO: CMAKE_BUILD_SHARED_LIBS    = 0
INFO: PYTHON_VERSION             = 3
INFO: BUILD_PYTHON               = 1
INFO: BUILD_PYTHON_LAYER         = 1
INFO: BUILD_MATLAB               = 0
INFO: PYTHON_EXE                 = "python"
INFO: RUN_TESTS                  = 0
INFO: RUN_LINT                   = 0
INFO: RUN_INSTALL                = 0
INFO: ============================================================
Der Befehl ""C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\..\..\VC\vcvarsall.bat"" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.
CMake Error at cmake/WindowsDownloadPrebuiltDependencies.cmake:40 (message):
  Could not find url for MSVC version = 1915 and Python version = 3.7.

Which makes sense as i can't seem to find any use of VS2017/py37 compiling on windows.
So would i just need to switch to VS2015?

1. Link to the Caffe.dlls where the C++ binaries lie.
   https://github.com/BVLC/caffe/issues/5053

2. Start using Caffe

I hope you get my problem, if you need more info i am eager to help.

[sandeshshrestha45]

@willyd
I successfully installed caffe without gpu in anaconda a following your instructions. But i cannot import it. I get the following error:

from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer File "C:\Users\San\Anaconda3\envs\caffe\lib\site-packages\caffe\pycaffe.py", line 13, in from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \ ImportError: DLL load failed: The specified module could not be found.

Could you please help me with this?

[bth5032]

I'm having the same error on Windows after installing caffe-cpu with anaconda using the following:

conda config --add channels anaconda
conda install caffe-cpu -c willyd

Python 3.6.8 |Anaconda, Inc.| (default, Feb 21 2019, 18:30:04) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\bobakh\AppData\Local\Continuum\anaconda3\lib\site-packages\caffe\__init__.py", line 1, in <module>
    from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer
  File "C:\Users\bobakh\AppData\Local\Continuum\anaconda3\lib\site-packages\caffe\pycaffe.py", line 13, in <module>
    from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \
ImportError: DLL load failed: The specified module could not be found.
>>>

[Banus]

With caffe-cpu I have the same problem of 'DLL load failed', under Windows 10 and conda Python 3.7.

Launching the caffe command from terminal, some debug DLLs are missing, e.g. UCRTBASED or VCRUNTIME14D. Dependency walker seems to suggest that the problem is in GLOG.DLL.

Unfortunately Caffe depends on this specific version, and trying to force a different one leads to the removal of the caffe-cpu package, so I wasn't able to verify if this is the source of the problem.

[WMF1997]

Hello @willyd :

I installed caffe and caffe-cpu via your conda packages successfully. And I want to ask a little question.

If I use a new C++ layer(NOT PYTHON LAYER) which is not in the origin caffe layers. for example, i use a triplet loss layer, the link of the triplet loss is here.
And I want to use this layer in a project. what should I do if I want to use this new C++/CUDA layer? (Is it right to put the .hpp, .cpp, .cu files to their related folders, and then just run the python code? (P.S. I use pycaffe))

yours,
@WMF1997

[willyd]

@WMF1997

Short answer: you can't.

Long answer: What you suggest is the right way to do it though. But you need to rebuild the conda packages by yourself. It could be as simple as:

1. Copy the files of the triplet loss layer as you suggest
2. Checkout this branch https://github.com/willyd/caffe/tree/windows
3. conda install conda-build
4. cd caffe\\conda && conda build .

I am not sure this will work since the conda recipes found in this branch might be out of date. I really don't have much time to dedicate to the caffe project these days. Otherwise, try to follow the standard building from source procedure.

[WMF1997]

Thank you very much~ @willyd

[HuangShiqing]

btw，if you just want to install a caffe-cpu on windows, It could be as simple as:
//I have try python27 and 35, all failed, only 36 succeed
conda create -n caffe36 python=3.6
conda activate caffe36
conda install -c willyd caffe-cpu

[lazylazypig]

@bth5032 Hi, have you sloved the 'import caffe' problem?

[bth5032]

@lazylazypig nope, switched to pytorch 😅

[sgnet91]

Hi @willyd ,
After having tried many solutions without success, I've tried your procedure but I'm still not able to import caffe.

[Kings77]

Hi @willyd i tried using the method that you suggest but i faced some problems. i installed it using the anaconda promp since im using the anaconda3. when i installed, it become an envs folder and not as the pkg. so i cant import caffe inside my work envs. what should i do?

I have published conda packages. Please install with these commands:

REM Create a new conda environment to install Caffe
REM You can use python 3.5, 3.6 or 3.7
conda create -n caffe python=3.7
REM activate the environment
REM Depending on your conda version you may have to use activate caffe
conda activate caffe
REM add the anaconda channel to resolve all dependencies
conda config --add channels anaconda
REM Install caffe with cuda support or install caffe-cpu if you do not want CUDA support
conda install caffe -c willyd

Currently only CUDA 9.0 is built but I will merge the files to build the packages once I have confirmation that the packages work.

[Alabenba]

@willyd
I successfully installed caffe without gpu in anaconda a following your instructions. But i cannot import it. I get the following error:
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver, NCCL, Timer File "C:\Users\San\Anaconda3\envscaffe\lib\site-packagescaffe\pycaffe.py", line 13, in from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, \ ImportError: DLL load failed: The specified module could not be found.

Could you please help me with this?

I have the same problem. Have you fixed it?

[sgnet91]

No, unfortunately not. I gave up the idea of using caffe and switched to keras. Good luck!

[liaojimzen]

So there is any idea to instead caffe?