Hi
When i want to import caffe i will face with this error:
Failed to include caffe_pb2, things might go wrong! Traceback (most recent call last): File "
Steps to reproduce
If you are having difficulty building Caffe or training a model, please ask the caffe-users mailing list. If you are reporting a build error that seems to be due to a bug in Caffe, please attach your build configuration (either Makefile.config or CMakeCache.txt) and the output of the make (or cmake) command.
Your system configuration
Operating system: win 10
Compiler: anaconda
CMakeCache.txt
CUDA version (if applicable):6
CUDNN version (if applicable):8
BLAS:
Python or MATLAB version (for pycaffe and matcaffe respectively):3.5
megh500
All 13 comments
To me this looks like you installed an incompatible python protobuf. You should install python protobuf 3.1.0.
willyd
on 18 May 2017
thanks
it seems it is not the last version i should install.
it downloaded it but it does not have anything for windows(setup file)is there any help?
megh500
on 19 May 2017
https://github.com/BVLC/caffe/tree/windows#using-the-python-interface
willyd
on 19 May 2017
willyd I have done the instruction in your link but still not working ):
Should I use make to compile it before ? In windows ?
stavBodik
on 14 Dec 2017
@stavBodik You need to compile caffe of course. On windows you can use the VS or Ninja Generators but not make. I personnally recommand ninja since it will build much faster than VS. However, it can be a bit confusing for new comers that are more used to VS.
If you want to beta test a very simple solution you can try my latest unofficial vcpkg port like so:
With VS 2017 and CUDA 9 installed open a cmd prompt and type:
git clone https://github.com/willyd/vcpkg.git
cd vcpkg
git checkout caffe
bootstrap-vcpkg.bat
# for a CPU only build
vcpkg install caffe --triplet x64-windows
# for a GPU build
vcpkg install caffe[cuda] --triplet x64-windows --featurepackages
This won't build the python bindings though, I will include them later.
willyd
on 14 Dec 2017
@stavBodik, I recently found another alternative to build Caffe on Windows though did not really check if Python bindings work: http://github.com/dividiti/ck-caffe . Seems like they try to provide a common front-end to build Caffe across Linux, Windows and MacOS. Following their instructions you should be able to build Caffe like this:
> pip install ck
> ck pull repo --url=https://github.com/dividiti/ck-caffe
> ck install package:lib-caffe-bvlc-master-cuda-universal --env.CAFFE_BUILD_PYTHON=ON
> ck run program:caffe
mlosab3
on 14 Dec 2017
Thanks guys !
The main problem is that caffe don't really give instructions for newb people(who use windows, and don't know what is make or using command line like in linux)... for example newb question :
"I heard Caffe is python framework for AI why we should compile it, it is python?"
Or at least caffe could say that we should compile and only then copy to site-packages or upload the pre -compiled version...
any way I Found solution , so before copying the python/caffe folder to site-packages ,
use this repository for simple compile and build caffe for windows :
https://github.com/BVLC/caffe/tree/windows/scripts
Just run "build_win.cmd" file with correct configurations, THIS SCRIPT WILL BUILD THE C++ PROJECTS(using cmake / ninja / visual studio cl.exe or whatever) AND WILL CREATE DLL FILES INSIDE "python/caffe" folder ...sounds better no ? ... Now you can copy this folder with the DLL's needed for python to run caffe into "site-packages" folder where Anaconda or Python installed in your windows , now you can test it by writing import caffe in python (:
stavBodik
on 14 Dec 2017
ck install package:lib-caffe-bvlc-master-cuda-universal --env.CAFFE_BUILD_PYTHON=ON
I ran this line in my MS VC2017 Command Prompt, I go these:
C:\Project Caffe>ck install package:lib-caffe-bvlc-master-cuda-universal --env.CAFFE_BUILD_PYTHON=ON
Resolving software dependencies ...
* Dependency 1 = python (Python for interfacing):
Resolved. CK environment UID = 11b4b1ba805a6f0b (detected version 3.6.5)
* Dependency 2 = cmake (cmake):
Resolved. CK environment UID = 18da984a66928577 (detected version 3.10.2)
* Dependency 3 = lib-boost (Boost library):
WARNING: no registered CK environment was found for "Boost library" dependency with tags="lib,boost" and setup={"host_os_uoa": "windows-64", "target_os_uoa": "windows-64", "target_os_bits": "64"} and version constraints ([1, 64, 0] <= v <= [])
Searching and installing CK software packages ...
* tags: lib,boost
* or tags:
* no tags:
More than one package found:
0) lib-boost-1.66.0-min-for-caffe Version 1.66.0-min-for-caffe (04254b0e8fd053ef)
1) lib-boost-1.66.0 Version 1.66.0 (0417ba98aa5ca4ad)
2) lib-boost-1.65.1-min-for-caffe Version 1.65.1-min-for-caffe (e8e0c4ee4898b84b)
3) lib-boost-1.65.1 Version 1.65.1 (8bb8a5b19b48120c)
4) lib-boost-1.64.0-min-for-caffe Version 1.64.0-min-for-caffe (cb7aa7d5bbd74d94)
5) lib-boost-1.64.0 Version 1.64.0 (c6a6991d5c951d20)
Select package number (or Enter to select 0): 1
Package found: lib-boost-1.66.0 (0417ba98aa5ca4ad)
Resolving software dependencies ...
* Dependency 1 = python (Unix Utils for Windows):
Resolved. CK environment UID = 11b4b1ba805a6f0b (detected version 3.6.5)
* Dependency 2 = cmake (cmake):
Resolved. CK environment UID = 18da984a66928577 (detected version 3.10.2)
* Dependency 3 = compiler (C++ compiler):
WARNING: no registered CK environment was found for "C++ compiler" dependency with tags="compiler,lang-cpp" and setup={"host_os_uoa": "windows-64", "target_os_uoa": "windows-64", "target_os_bits": "64"}
Trying to automatically detect required software ...
1) Checking if "GNU C compiler" (compiler.gcc / 3a8a82fa40bf992d) is installed ...
Searching for GNU C compiler (gcc.exe) to automatically register in the CK - it may take some time, please wait ...
* Searching in C:\Program Files ...
* Searching in C:\Program Files (x86) ...
* Searching in C:\Users\aa ...
* Searching in C:\MinGW ...
Search completed in 2.8 secs. Found 2 target files (may be pruned) ...
Detecting and sorting versions (ignore some work output) ...
* C:\Users\aa\CK-TOOLS\mingw-gcc-6.3.0-6.3.0-windows-64\mingw64\bin\gcc.exe (Version 6.3.0)
* C:\MinGW\bin\gcc.exe (Version 6.3.0)
Registering software installations found on your machine in the CK:
(HINT: enter -1 to force CK package installation)
0) Version 6.3.0 - C:\Users\aa\CK-TOOLS\mingw-gcc-6.3.0-6.3.0-windows-64\mingw64\bin\gcc.exe
1) Version 6.3.0 - C:\MinGW\bin\gcc.exe
Please, select the number of any above installation or press Enter to select 0: 1
Registering in the CK (C:\MinGW\bin\gcc.exe) ...
Software entry found: compiler.gcc (3a8a82fa40bf992d)
Attempting to detect version automatically (if supported) ...
Detected version: 6.3.0
Searching if environment already exists using:
- Tags: compiler,gcc,lang-c,lang-cpp,host-os-windows-64,target-os-windows-64,64bits,v6,v6.3,v6.3.0
Environment with above tags is not yet registered in CK ...
Traceback (most recent call last):
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 9231, in
r=access(sys.argv[1:])
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 9187, in access
rr=perform_action(i)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 3321, in perform_action
return a(i)
File "C:\Users\aa\CK\ck-env\module\package\module.py", line 689, in install
rx=ck.access(env_resolve_action_dict)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 9187, in access
rr=perform_action(i)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 3321, in perform_action
return a(i)
File "C:\Users\aa\CK\ck-env\module\env\module.py", line 1459, in resolve
rx=set(ii)
File "C:\Users\aa\CK\ck-env\module\env\module.py", line 714, in set
'deps':cdeps})
File "C:\Users\aa\CK\ck-env\module\env\module.py", line 2195, in internal_install_package
rx=ck.access(vv)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 9187, in access
rr=perform_action(i)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 3321, in perform_action
return a(i)
File "C:\Users\aa\CK\ck-env\module\package\module.py", line 689, in install
rx=ck.access(env_resolve_action_dict)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 9187, in access
rr=perform_action(i)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 3321, in perform_action
return a(i)
File "C:\Users\aa\CK\ck-env\module\env\module.py", line 1459, in resolve
rx=set(ii)
File "C:\Users\aa\CK\ck-env\module\env\module.py", line 515, in set
ry=ck.access(ii)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 9187, in access
rr=perform_action(i)
File "C:\Users\aaAppData\Local\Programs\Python\Python36\lib\site-packages\ck\kernel.py", line 3321, in perform_action
return a(i)
File "C:\Users\aa\CK\ck-env\module\soft\module.py", line 2012, in check
rz=setup(ii)
File "C:\Users\aa\CK\ck-env\module\soft\module.py", line 1053, in setup
rx=cs.setup(ii)
File "C:\Users\aa\CK\ck-env\soft\compiler.gcc\customize.py", line 185, in setup
if path_lib:
UnboundLocalError: local variable 'path_lib' referenced before assignment
captkkk
on 7 Jun 2018
Hi @captkkk - thanks for reporting this bug - I believe I just fixed it. Can you please try to update repositories and try again:
$ ck pull repo:ck-env
$ ck install package:lib-caffe-bvlc-master-cuda-universal --env.CAFFE_BUILD_PYTHON=ON
BTW, I don't know if there is a support in latest Caffe for Boost 1.66.0, so I would suggest you to install "lib-boost-1.65.1-min-for-caffe", i.e.
$ ck install package:lib-boost-1.65.1-min-for-caffe
and then restart Caffe build ...
Also, I didn't check the latest Caffe with CK workflow on Windows for a few months, so will check it too and will tell you if I encounter some issues ...
Thanks!
gfursin
on 7 Jun 2018
@captkkk - I just managed to build package:lib-caffe-bvlc-master-cuda-universal via CK on a clean environment so it should be working fine.
However, note that I still couldn't build Caffe (this package) with the latest Microsoft C compiler "19.11.25547" . I remember someone already reported issues with that so I hope someone will still fix cmake for Caffe. I used "Microsoft C compiler 19.00.24215.1" and it worked fine.
Also, I had to fix Boost Include path and convert to CYGWIN format. You just need to pull latest changes and refresh CK virtual env:
$ ck pull all
$ ck refresh env
Here are my dependencies via CK:
Resolving software dependencies ...
*** Dependency 1 = python (Python for interfacing):
Resolved. CK environment UID = 4836f355c8844a3a (detected version 3.6.3)
*** Dependency 2 = cmake (cmake):
Resolved. CK environment UID = 6e2709e436e8e513 (detected version 3.10.2)
*** Dependency 3 = lib-boost (Boost library):
Resolved. CK environment UID = 5df21b65fa8e4329 (version 1.65.1-min-for-caffe)
*** Dependency 4 = lib-gflags (GFlags library):
Resolved. CK environment UID = 7415db0a41823247 (version 2.2.0)
*** Dependency 5 = lib-glog (GLog library):
Resolved. CK environment UID = 2f28e7f81b141c6a (version 0.3.5-a6a166d)
*** Dependency 6 = compiler (C++ compiler):
Resolved. CK environment UID = 46f083a96b220732 (detected version 19.00.24215.1)
*** Dependency 7 = compiler-cuda (CUDA compiler >=7.0):
Resolved. CK environment UID = 853e3ce658946bbb (detected version 9.1.85)
*** Dependency 8 = lib-opencv (OpenCV library):
Resolved. CK environment UID = 479ba845b8095d0f (version 3.3.0)
*** Dependency 9 = lib-lmdb (LMDB library):
Resolved. CK environment UID = e6c9451b6cbb09b9 (version 0.9.21-60d5002)
*** Dependency 10 = lib-hdf5 (HDF5 library):
Resolved. CK environment UID = ec98e042b996686c (version 1.10.1)
*** Dependency 11 = lib-blas (BLAS library):
Resolved. CK environment UID = 45dbbf46e4f93051 (version 0.3.0-939452ea)
*** Dependency 12 = lib-protobuf-host (ProtoBuf host compiler):
Resolved. CK environment UID = 867904424599c038 (version 3.5.1)
-----------------------------------
Detected GIT revision: 509dae05
Detected GIT date time of last commit: Wed Jan 3 10:20:53 2018 +0100
You still need to install some missing deps for your Python via pip (numpy, scipy, protobuf, google, scikit-image) and then Python Caffe seems to be working (at least I can do "import caffe" but I didn't have time to check it further). You can run CK virtual env for a test (after you build Caffe via CK):
D:\>ck virtual env --tags=python
Warning: you are in a new shell with a pre-set CK environment. Enter "exit" to return to the original one!
D:\>ck virtual env --tags=lib,caffe
Warning: you are in a new shell with a pre-set CK environment. Enter "exit" to return to the original one!
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.4.4
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'
D:\>python
Python 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>>
Hope it's of any help. If you have further questions, feel free to get in touch with the CK users - we continue improving packages for different AI frameworks based on user feedback: https://github.com/ctuning/ck/wiki/Contacts
Cheers!
gfursin
on 7 Jun 2018
Guys, have to say thanks many times for your supports, I have moved forward a little bit, then hit this wall:
C:>ck virtual env --tags=lib,caffe
WARNING: no registered CK environment was found for required software dependency with tags="lib,caffe"
Trying to automatically detect required software ...
1) Checking if "BVLC Caffe framework" (lib.caffe / 04ab08137582bd03) is installed ...
Searching for BVLC Caffe framework (caffe.exe) to automatically register in the CK - it may take some time, please wait ...
* Searching in C:\Program Files ...
* Searching in C:\Program Files (x86) ...
* Searching in C:\Users\aa ...
Search completed in 2.9 secs. Found 0 target files (may be pruned) ...
(warning during intermediate step: software was not automatically found on your system! Please, install it and re-try again!)
No software auto-detection scripts found for this software in CK :( ...
Checked following related CK soft entries:
* lib.caffe
Searching and installing CK software packages ...
* tags: lib,caffe
* or tags:
* no tags:
More than one package found:
0) lib-caffe-bvlc-opencl-libdnn-clblast-universal-20171015 Version master-20171015 (RECOMMENDED for Linux and Android, 84ee4c98a58a85b5)
1) lib-caffe-bvlc-opencl-clblast-universal-20171015 Version master-20171015 (RECOMMENDED for Linux and Android, 3691441b8aa92bd1)
2) lib-caffe-bvlc-opencl-clblast-universal Version master (RECOMMENDED, df78a1afc47a1962)
3) lib-caffe-bvlc-master-cuda-universal-20171020 Version master-20171020 (build stable Caffe CUDA from master snapshot 20171020, a7da0d585da04bf4)
4) lib-caffe-bvlc-opencl-viennacl-universal-20171015 Version master-20171015 (a9e57a9fdf4df2bc)
5) lib-caffe-bvlc-master-cpu-universal-20171013 Version master-20171013 (build stable Caffe CPU from master snapshot 20171013, af7cfba74b1ea1d6)
6) lib-caffe-nvidia-0.17-cudnn-universal Version nvidia-0.17 (48c184ebc296318e)
7) lib-caffe-nvidia-0.16-cudnn-universal Version nvidia-0.16 (46ba683761b6d554)
8) lib-caffe-nvidia-0.16-cuda-universal Version nvidia-0.16 (4046d444abfc296a)
9) lib-caffe-intel-master-cpu Version master (4f48b2e77da739cc)
10) lib-caffe-bvlc-opencl-viennacl-universal Version master (4d2c936d65a4213c)
11) lib-caffe-bvlc-opencl-libdnn-viennacl-universal Version master (c412f85ecc2bd6c9)
12) lib-caffe-bvlc-opencl-libdnn-viennacl-bin-win Version 5a1422a93a842bd6 (979053678e07463a)
13) lib-caffe-bvlc-opencl-libdnn-mixed-precision Version trunk (679b494ce6947086)
14) lib-caffe-bvlc-opencl-libdnn-cuda-universal Version opencl (b880fbe45f1cf23d)
15) lib-caffe-bvlc-opencl-libdnn-clblast-universal-tune Version trunk (d53e4ec2b0db1b5d)
16) lib-caffe-bvlc-opencl-libdnn-clblast-universal Version trunk (6d40548fa5d00956)
17) lib-caffe-bvlc-opencl-libdnn-clblas-universal Version trunk (eabb571a9fcdef0f)
18) lib-caffe-bvlc-opencl-cpu-universal Version trunk (9fa7bd782a74492d)
19) lib-caffe-bvlc-opencl-clblast-universal-tune Version master (1f7add8fbdb483a0)
20) lib-caffe-bvlc-opencl-clblas-universal Version master (d4aaaced3540efbb)
21) lib-caffe-bvlc-master-cudnn-universal-20171020 Version master (build stable Caffe cuDNN from master snapshot 20171020, 7a44d98990abd5e8)
22) lib-caffe-bvlc-master-cudnn-universal Version master (c9f484014ba7d5b0)
23) lib-caffe-bvlc-master-cuda-universal Version master (dfa03d72a1f755c4)
24) lib-caffe-bvlc-master-cpu-universal Version master (build Caffe CPU from master, b39f40abfd34f10b)
25) lib-caffe-bvlc-master-cpu-bin-win Version 1c1be4c4592468c7 (download pre-built Caffe, 1569409b9e2d2ae0)
Select package number (or Enter to select 0): 24
Package found: lib-caffe-bvlc-master-cpu-universal (b39f40abfd34f10b)
Resolving software dependencies ...
* Dependency 1 = python (Python for interfacing):
Resolved. CK environment UID = 11b4b1ba805a6f0b (detected version 3.6.5)
* Dependency 2 = cmake (cmake):
Resolved. CK environment UID = 18da984a66928577 (detected version 3.10.2)
* Dependency 3 = compiler (C++ compiler):
* Visual Studio 2017 Developer Command Prompt v15.7.3
* Copyright (c) 2017 Microsoft Corporation
[vcvarsall.bat] Environment initialized for: 'x64'
Resolved. CK environment UID = 428aac88529d3d81 (detected version 19.14.26430)
* Dependency 4 = lib-gflags (GFlags library):
Resolved. CK environment UID = d83d4741c58adff6 (version 2.2.0)
* Dependency 5 = lib-glog (GLog library):
Resolved. CK environment UID = 048a7244e63b1c75 (version 0.3.5-a6a166d)
* Dependency 6 = lib-lmdb (LMDB library):
WARNING: no registered CK environment was found for "LMDB library" dependency with tags="lib,lmdb" and setup={"host_os_uoa": "windows-64", "target_os_uoa": "windows-64", "target_os_bits": "64"}
Trying to automatically detect required software ...
1) Checking if "LMDB library" (lib.lmdb / a249f58927ceefa5) is installed ...
Searching for LMDB library (lmdb.lib) to automatically register in the CK - it may take some time, please wait ...
* Searching in C:\Program Files ...
* Searching in C:\Program Files (x86) ...
* Searching in C:\Users\aa ...
Search completed in 0.8 secs. Found 0 target files (may be pruned) ...
(warning during intermediate step: software was not automatically found on your system! Please, install it and re-try again!)
No software auto-detection scripts found for this software in CK :( ...
Checked following related CK soft entries:
* lib.lmdb
Searching and installing CK software packages ...
* tags: lib,lmdb
* or tags:
* no tags:
More than one package found:
0) lib-lmdb-0.9.21 Version 0.9.21 (48744027b0254a13)
1) lib-lmdb-master-universal Version master (5f41ddacc856c14d)
Select package number (or Enter to select 0): 1
Package found: lib-lmdb-master-universal (5f41ddacc856c14d)
Resolving software dependencies ...
* Dependency 1 = compiler (C++ compiler):
* Visual Studio 2017 Developer Command Prompt v15.7.3
* Copyright (c) 2017 Microsoft Corporation
[vcvarsall.bat] Environment initialized for: 'x64'
Resolved. CK environment UID = 428aac88529d3d81 (detected version 19.14.26430)
Searching if CK environment for this package already exists using:
- Tags: compiled-by-mcl,compiled-by-mcl-19.14.26430,lib,lmdb,vtrunk,vmaster,channel-dev,host-os-windows-64,target-os-windows-64,64bits
- Dependency: compiler=428aac88529d3d81
CK environment not found for this package ...
* Installation path used: C:\Users\aa\CK-TOOLS\lib-lmdb-master-mcl-19.14.26430-windows-64
Resolving software dependencies ...
* Dependency 1 = compiler (C++ compiler):
* Visual Studio 2017 Developer Command Prompt v15.7.3
* Copyright (c) 2017 Microsoft Corporation
[vcvarsall.bat] Environment initialized for: 'x64'
Resolved. CK environment UID = 428aac88529d3d81 (detected version 19.14.26430)
Installing to C:\Users\aa\CK-TOOLS\lib-lmdb-master-mcl-19.14.26430-windows-64
* Visual Studio 2017 Developer Command Prompt v15.7.3
* Copyright (c) 2017 Microsoft Corporation
[vcvarsall.bat] Environment initialized for: 'x64'
* Visual Studio 2017 Developer Command Prompt v15.0
* Copyright (c) 2017 Microsoft Corporation
[ERROR:vcvars.bat] Version '14.14.26428' is not valid; directory does not exist
[ERROR:VsDevCmd.bat] * VsDevCmd.bat encountered errors. Environment may be incomplete and/or incorrect. *
[ERROR:VsDevCmd.bat] In an uninitialized command prompt, please 'set VSCMD_DEBUG=[value]' and then re-run
[ERROR:VsDevCmd.bat] vsdevcmd.bat [args] for additional details.
[ERROR:VsDevCmd.bat] Where [value] is:
[ERROR:VsDevCmd.bat] 1 : basic debug logging
[ERROR:VsDevCmd.bat] 2 : detailed debug logging
[ERROR:VsDevCmd.bat] 3 : trace level logging. Redirection of output to a file when using this level is recommended.
[ERROR:VsDevCmd.bat] Example: set VSCMD_DEBUG=3
[ERROR:VsDevCmd.bat] vsdevcmd.bat > vsdevcmd.trace.txt 2>&1
Cloning package from 'https://github.com/LMDB/lmdb/' ...
Cloning into 'src'...
remote: Counting objects: 8016, done.
remote: Total 8016 (delta 0), reused 0 (delta 0), pack-reused 8015Receiving objects: 100% (8016/8016), 1.53 MiB | 923.00 KiB/s
Receiving objects: 100% (8016/8016), 1.85 MiB | 1.07 MiB/s, done.
Resolving deltas: 100% (3459/3459), done.
Cleaning ...
Executing extra script ...
lmdb.h
midl.h
2 file(s) copied.
Building static library ...
Could Not Find C:\Users\aa\CK-TOOLS\lib-lmdb-master-mcl-19.14.26430-windows-64\src\libraries\liblmdb*.obj
Could Not Find C:\Users\aa\CK-TOOLS\lib-lmdb-master-mcl-19.14.26430-windows-64\src\libraries\liblmdb\lmdb.lib
Executing cl /Ox /MD /c /DWINDOWS /GS- mdb.c midl.c
Microsoft (R) C/C++ Optimizing Compiler Version 19.14.26430 for x64
Copyright (C) Microsoft Corporation. All rights reserved.
mdb.c
mdb.c(55): error C2059: syntax error: '('
mdb.c(61): error C2143: syntax error: missing '{' before ''
mdb.c(68): error C2059: syntax error: '('
mdb.c(75): error C2143: syntax error: missing '{' before ''
mdb.c(77): error C2059: syntax error: '('
mdb.c(79): error C2143: syntax error: missing '{' before '*'
mdb.c(4339): error C2063: 'NtCreateSection': not a function
mdb.c(4346): error C2063: 'NtMapViewOfSection': not a function
mdb.c(4350): error C2063: 'NtClose': not a function
mdb.c(4704): error C2065: 'NtCloseFunc': undeclared identifier
mdb.c(4704): error C2059: syntax error: ')'
mdb.c(4707): error C2065: 'NtMapViewOfSectionFunc': undeclared identifier
mdb.c(4707): error C2059: syntax error: ')'
mdb.c(4710): error C2065: 'NtCreateSectionFunc': undeclared identifier
mdb.c(4710): error C2059: syntax error: ')'
mdb.c(8314): warning C4333: '>>': right shift by too large amount, data loss
midl.c
Generating Code...
Building failed!
The system cannot find the batch label specified - err
Cleaning obj directory ...
Setting up environment for installed package ...
(full path = C:\Users\aa\CK-TOOLS\lib-lmdb-master-mcl-19.14.26430-windows-64\lib\lmdb.lib)
Software entry found: lib.lmdb (a249f58927ceefa5)
Resolving software dependencies ...
* Dependency 1 = compiler (C++ compiler):
* Visual Studio 2017 Developer Command Prompt v15.7.3
* Copyright (c) 2017 Microsoft Corporation
[vcvarsall.bat] Environment initialized for: 'x64'
Resolved. CK environment UID = 428aac88529d3d81 (detected version 19.14.26430)
Detected GIT revision: 86a90ca
Detected GIT date time of last commit: Wed May 2 17:05:29 2018 +0100
Searching if environment already exists using:
- Tags: lib,lmdb,compiled-by-mcl,compiled-by-mcl-19.14.26430,vtrunk,vmaster,channel-dev,host-os-windows-64,target-os-windows-64,64bits,v0,v0.0
- Dependency: compiler=428aac88529d3d81
Environment with above tags is not yet registered in CK ...
CK error: [env] can't find root dir of this installation!
captkkk
on 11 Jun 2018
@captkkk - I reproduced your problem - seems that the latest GitHub version of LMDB is now broken for Windows :( . I will open an issue there. In the mean time, you can install stable LMDB 0.9.21 (I just checked and it works on Windows):
$ ck install package:lib-lmdb-0.9.21
then restart your Caffe build ...
BTW, I just updated my Visual Studio compiler and have the same version as yours now:
D:\Work1\CK\ck-repos\ck-env\package>ck show env --tags=mcl
Env UID: Target OS: Bits: Name: Version: Tags:
76e1fafe5441382e windows-64 64 Microsoft C compiler 19.14.26430 64bits,compiler,host-os-windows-64,lang-c,lang-cpp,mcl,microsoft,target-os-windows-64,v19,v19.14,v19.14.26430
aeb9ba90458fa1f5 windows-64 64 Microsoft C compiler 19.00.24215.1 64bits,compiler,host-os-windows-64,lang-c,lang-cpp,mcl,microsoft,target-os-windows-64,v19,v19.0,v19.0.24215,v19.0.24215.1
1a563a7534c74a16 windows-64 64 Microsoft C compiler 18.00.40629 64bits,compiler,host-os-windows-64,lang-c,lang-cpp,mcl,microsoft,target-os-windows-64,v18,v18.0,v18.0.40629
Will also try to check later if it's possible to build Caffe with this version ...
Cheers!
gfursin
on 11 Jun 2018
@captkkk - I also removed sm_2x CUDA arch from cmake via patch in the CK package since CUDA 9.x complains about that. You can update your CK environment using:
$ ck pull all
Also notice that latest VS 2017 still has issues with CUDA 9+. I managed to fix part of it using the solution described here (changing host_config.h): https://stackoverflow.com/questions/43745099/using-cuda-with-visual-studio-2017. You can test if your nvcc works with you VS 2017 as following:
$ ck compile program:tool-print-cuda-devices
$ ck run program:tool-print-cuda-devices
However, I still didn't manage to compile Caffe on Windows with the latest VS 2017 and CUDA 9.2:
CMake configure flags:
-G"Visual Studio 15 2017 Win64" -DCPU_ONLY:BOOL=0 -DBLAS=Open -DUSE_LMDB=ON -DUSE_LEVELDB=OFF -DUSE_PREBUILT_DEPENDENCIES=OFF -DBUILD_SHARED_LIBS:BOOL=0 -DBUILD_python=ON -DBUILD_matlab:BOOL=0
-DGFLAGS_INCLUDE_DIR="D:\Work1\CK\ck-repos\local\env\8ea848959503f379\install\include" -DGFLAGS_LIBRARY="D:\Work1\CK\ck-repos\local\env\8ea848959503f379\install\Lib\gflags.lib" -DCUDA_TOOLKIT_ROOT
_DIR="C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2" -DCUDA_ARCH_NAME="Auto" -DCUDA_NVCC_FLAGS="-D_FORCE_INLINES -Wno-deprecated-gpu-targets" -DGLOG_INCLUDE_DIR="D:\Work1\CK\ck-repos\loca
l\env\f694f9f8e88db4ca\install\include" -DGLOG_LIBRARY="D:\Work1\CK\ck-repos\local\env\f694f9f8e88db4ca\install\lib\glog.lib" -DLMDB_INCLUDE_DIR="D:/Work1/CK/ck-repos/local/env/87a10f1d23e19599/incl
ude" -DLMDB_LIBRARIES="D:/Work1/CK/ck-repos/local/env/87a10f1d23e19599/lib\lmdb.lib" -DCMAKE_BUILD_TYPE:STRING=Release -DPROTOBUF_DIR="D:\Work1\CK\ck-repos\local\env\da81aa2b9df44eb5\install\cmake"
-DHDF5_ROOT_DIR="D:\Work1\CK\ck-repos\local\env\0a4748b3a288f7b9\install\cmake" -DHDF5_INCLUDE_DIRS="D:\Work1\CK\ck-repos\local\env\0a4748b3a288f7b9\install\include" -DHDF5_LIBRARIES="D:\Work1\CK\
ck-repos\local\env\0a4748b3a288f7b9\install\lib\hdf5.lib" -DHDF5_HL_LIBRARIES="D:\Work1\CK\ck-repos\local\env\0a4748b3a288f7b9\install\lib\hdf5_hl.lib" -DOpenBLAS_INCLUDE_DIR="D:\Work1\CK\ck-repos\l
ocal\env\90e54479b47a7ff2\install\include" -DOpenBLAS_LIB="D:\Work1\CK\ck-repos\local\env\90e54479b47a7ff2\install/lib\libopenblas.dll.a" -DBoost_ADDITIONAL_VERSIONS="1.62" -DBoost_NO_SYSTEM_PATHS=
ON -DBOOST_ROOT=D:\Work1\CK\ck-repos\local\env\51f0e2555b34d629\install -DBOOST_INCLUDEDIR="D:/Work1/CK/ck-repos/local/env/51f0e2555b34d629/install/include" -DBOOST_LIBRARYDIR="D:\Work1\CK\ck-repos
\local\env\51f0e2555b34d629\install\lib" -DBoost_INCLUDE_DIR="D:/Work1/CK/ck-repos/local/env/51f0e2555b34d629/install/include" -DBoost_LIBRARY_DIR="D:\Work1\CK\ck-repos\local\env\51f0e2555b34d629\in
stall\lib" -DBoost_PYTHON_LIBRARY_DEBUG:FILEPATH=D:\Work1\CK\ck-repos\local\env\51f0e2555b34d629\install\lib\boost_python3-mt-gd.lib -DBoost_PYTHON_LIBRARY_RELEASE:FILEPATH=D:\Work1\CK\ck-repos\local
\env\51f0e2555b34d629\install\lib\boost_python3-mt.lib -DOpenCV_DIR="D:\Work1\CK\ck-repos\local\env\24662c472e4094a2\install" -DOpenCV_LIB_PATH="D:\Work1\CK\ck-repos\local\env\24662c472e4094a2\insta
ll\lib"
Configuring ...
-- Boost version: 1.65.1
-- Found the following Boost libraries:
-- system
-- thread
-- filesystem
-- chrono
-- date_time
-- Found gflags (include: D:/Work1/CK/ck-repos/local/env/8ea848959503f379/install/Include, library: gflags_shared)
-- Found glog (include: D:/Work1/CK/ck-repos/local/env/f694f9f8e88db4ca/install/include, library: D:\Work1\CK\ck-repos\local\env\f694f9f8e88db4ca\install\lib\glog.lib)
-- Found PROTOBUF Compiler: D:/Work1/CK/ck-repos/local/env/da81aa2b9df44eb5/install/bin/protoc.exe
-- Could NOT find LMDB (missing: LMDB_DIR)
-- Found lmdb (include: D:/Work1/CK/ck-repos/local/env/87a10f1d23e19599/include, library: D:/Work1/CK/ck-repos/local/env/87a10f1d23e19599/lib\lmdb.lib)
-- CUDA detected: 9.2
-- Found cuDNN: ver. 7.1.4 found (include: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/include, library: C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/lib/x64/cudnn.lib)
-- Added CUDA NVCC flags for: sm_50
-- OpenCV found (D:/Work1/CK/ck-repos/local/env/24662c472e4094a2/install/lib)
-- Found OpenBLAS libraries: D:/Work1/CK/ck-repos/local/env/90e54479b47a7ff2/install/lib/libopenblas.dll.a
-- Found OpenBLAS include: D:/Work1/CK/ck-repos/local/env/90e54479b47a7ff2/install/include
-- NumPy ver. 1.13.3 found (include: C:/Users/fgg/Anaconda3/lib/site-packages/numpy/core/include)
-- Boost version: 1.65.1
-- Found the following Boost libraries:
-- python
--
-- ******************* Caffe Configuration Summary *******************
-- General:
-- Version : 1.0.0
-- Git : 1.0-111-g509dae05-dirty
-- System : Windows
-- C++ compiler : C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.14.26428/bin/Hostx86/x64/cl.exe
-- Release CXX flags : /MD /O2 /Ob2 /DNDEBUG /DWIN32 /D_WINDOWS /W3 /GR /EHsc
-- Debug CXX flags : /MDd /Zi /Ob0 /Od /RTC1 /DWIN32 /D_WINDOWS /W3 /GR /EHsc
-- Build type : Release
--
-- BUILD_SHARED_LIBS : 0
-- BUILD_python : ON
-- BUILD_matlab : 0
-- BUILD_docs :
-- CPU_ONLY : 0
-- USE_OPENCV : ON
-- USE_LEVELDB : OFF
-- USE_LMDB : ON
-- USE_NCCL : OFF
-- ALLOW_LMDB_NOLOCK : OFF
--
-- Dependencies:
-- BLAS : Yes (Open)
-- Boost : Yes (ver. 1.65)
-- glog : Yes
-- gflags : Yes
-- protobuf : Yes (ver. 3.1.0)
-- lmdb : Yes (ver. 0.9.21)
-- OpenCV : Yes (ver. 3.3.0)
-- CUDA : Yes (ver. 9.2)
--
-- NVIDIA CUDA:
-- Target GPU(s) : Auto
-- GPU arch(s) : sm_50
-- cuDNN : Yes (ver. 7.1.4)
--
-- Python:
-- Interpreter : C:/Users/fgg/Anaconda3/python.exe (ver. 3.6.3)
-- Libraries : C:/Users/fgg/Anaconda3/libs/python36.lib (ver 3.6.3)
-- NumPy : C:/Users/fgg/Anaconda3/lib/site-packages/numpy/core/include (ver 1.13.3)
--
-- Install:
-- Install path : D:/Work1/CK/ck-repos/local/env/6f47090d91411c9a/install
--
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Work1/CK/ck-repos/local/env/6f47090d91411c9a/obj
**************************************************************
Building using Visual Studio ...
Microsoft (R) Build Engine version 15.7.179.6572 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.
Build started 11/06/2018 17:39:59.
Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\install.vcxproj" on node 1 (default targets).
Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\install.vcxproj" (1) is building "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\ZERO_CHECK.vcxproj" (2) on node 1 (default targets).
InitializeBuildStatus:
Creating "x64\Release\ZERO_CHECK\ZERO_CHECK.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
CustomBuild:
All outputs are up-to-date.
FinalizeBuildStatus:
Deleting file "x64\Release\ZERO_CHECK\ZERO_CHECK.tlog\unsuccessfulbuild".
Touching "x64\Release\ZERO_CHECK\ZERO_CHECK.tlog\ZERO_CHECK.lastbuildstate".
Done Building Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\ZERO_CHECK.vcxproj" (default targets).
Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\install.vcxproj" (1) is building "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\ALL_BUILD.vcxproj" (3) on node 1 (default targets).
Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\ALL_BUILD.vcxproj" (3) is building "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\src\caffe\caffe.vcxproj" (4) on node 1 (default ta
rgets).
Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\src\caffe\caffe.vcxproj" (4) is building "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\src\caffe\caffeproto.vcxproj" (5) on node 1
(default targets).
InitializeBuildStatus:
Creating "caffeproto.dir\Release\caffeproto.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
CustomBuild:
All outputs are up-to-date.
ClCompile:
All outputs are up-to-date.
Lib:
All outputs are up-to-date.
caffeproto.vcxproj -> D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\lib\Release\caffeproto.lib
FinalizeBuildStatus:
Deleting file "caffeproto.dir\Release\caffeproto.tlog\unsuccessfulbuild".
Touching "caffeproto.dir\Release\caffeproto.tlog\caffeproto.lastbuildstate".
Done Building Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\src\caffe\caffeproto.vcxproj" (default targets).
InitializeBuildStatus:
Touching "caffe.dir\Release\caffe.tlog\unsuccessfulbuild".
CustomBuild:
Building NVCC (Device) object src/caffe/CMakeFiles/cuda_compile_1.dir/layers/Release/cuda_compile_1_generated_absval_layer.cu.obj
CMake Error at cuda_compile_1_generated_absval_layer.cu.obj.Release.cmake:219 (message):
Error generating
D:/Work1/CK/ck-repos/local/env/6f47090d91411c9a/obj/src/caffe/CMakeFiles/cuda_compile_1.dir/layers/Release/cuda_compile_1_generated_absval_layer.cu.obj
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(171,5): error MSB6006: "cmd.exe" exited with code 1. [D:\Work1\CK\ck-repos\local\env
\6f47090d91411c9a\obj\src\caffe\caffe.vcxproj]
Done Building Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\src\caffe\caffe.vcxproj" (default targets) -- FAILED.
Done Building Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\ALL_BUILD.vcxproj" (default targets) -- FAILED.
Done Building Project "D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\install.vcxproj" (default targets) -- FAILED.
Build FAILED.
"D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\install.vcxproj" (default target) (1) ->
"D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\ALL_BUILD.vcxproj" (default target) (3) ->
"D:\Work1\CK\ck-repos\local\env\6f47090d91411c9a\obj\src\caffe\caffe.vcxproj" (default target) (4) ->
(CustomBuild target) ->
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets(171,5): error MSB6006: "cmd.exe" exited with code 1. [D:\Work1\CK\ck-repos\local\e
nv\6f47090d91411c9a\obj\src\caffe\caffe.vcxproj]
0 Warning(s)
1 Error(s)
Time Elapsed 00:00:03.95
My solution is to use previous version of VS as was suggested at different forums. If anyone knows another solution, please tell me so that I could update CK packages. Thanks!
gfursin
on 11 Jun 2018
Related issues
weather319
·
3Comments
hawklucky
·
3Comments
malreddysid
·
3Comments
LarsHH
·
3Comments
prathmeshrmadhu
·
3Comments
Most helpful comment
@stavBodik, I recently found another alternative to build Caffe on Windows though did not really check if Python bindings work: http://github.com/dividiti/ck-caffe . Seems like they try to provide a common front-end to build Caffe across Linux, Windows and MacOS. Following their instructions you should be able to build Caffe like this: