Incubator-mxnet: cannot compile Amalgamation

[ascust]

I am trying to compile Amalgamation, but I somehow failed. I have set the path for blas, so it was going well. However, it told me cuda.h was not found. This made me confusing. Do we really need cuda to compile this? I mean not every platform has cuda lib. Anyway, I added cuda into the env variable, but I got the problem like

In file included from mxnet_predict-all.cc:32:
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:84:26: error: expected unqualified-id
typedef struct __align__(2) {
                         ^
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:84:26: error: expected ')'
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:84:25: note: to match this '('
typedef struct __align__(2) {
                        ^
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:86:3: error: C++ requires a type specifier for all
      declarations
} __half;
  ^
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:88:26: error: expected unqualified-id
typedef struct __align__(4) {
                         ^
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:88:26: error: expected ')'
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:88:25: note: to match this '('
typedef struct __align__(4) {
                        ^
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:90:3: error: C++ requires a type specifier for all
      declarations
} __half2;
  ^
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:93:9: error: unknown type name '__half'
typedef __half half;
        ^
/Developer/NVIDIA/CUDA-7.5/include/cuda_fp16.h:94:9: error: unknown type name '__half2'
typedef __half2 half2;

I tried this on MAC OS as well as on Ubuntu. According to http://www.liuxiao.org/page/2/ . It seems it is not that complicated. Anyone has any idea?

[piiswrong]

You should need cuda. Make sure use_cuda is turned off.

[ascust]

I did add the cuda path, it was correct. Then I got the error as I showed above. You said make sure use_cuda is turned off. Do you mean in "mxnet_predict0.cc"? I did not change it, and it is like:

#define MSHADOW_FORCE_STREAM
#ifndef MSHADOW_USE_CBLAS
#define MSHADOW_USE_CBLAS   1
#endif
#define MSHADOW_USE_CUDA    0
#define MSHADOW_USE_MKL     0
#define MSHADOW_RABIT_PS    0
#define MSHADOW_DIST_PS     0

#if defined(__ANDROID__) || defined(__MXNET_JS__)
#define MSHADOW_USE_SSE         0
#endif

#define MXNET_USE_OPENCV    0
#define MXNET_PREDICT_ONLY  1
#define DISABLE_OPENMP 1

@piiswrong
By the way, do I need to compile mxnet first? in order to compile Amalgamation

[ascust]

Since I suspected it could because of the version of CUDA, I tried another one CUDA 7.0 (The previous one was CUDA 7.5). Now I got another problem,

No such file or directory #include <cuda_fp16.h>

After looking it through, it seems this file is only in CUDA7.5. It is so confusing. Do I have to use CUDA 7.5 strictly? Since this Amalgamation is like a platform independent framework and it's only dependency is OpenBlas, why would we need the other things like cuda. According to the README, after assigning the openblas path, the only thing is make. So anything I have missed? @tqchen @piiswrong

[wqfan]

I met a similar problem. I first complied MXNet without using CUDA (I use a Macbook without NVIDIA GPU), then I try to compile the Amalgamation. I got the following error:

Generating amalgamation to  mxnet_predict-all.cc
python ./amalgamation.py mxnet_predict0.d mxnet_predict0.cc mxnet_predict-all.cc 0
Not processed: mxnet_predict0.o:
c++ -std=c++11 -Wno-unknown-pragmas -Wall -I/usr/local/opt/openblas/include -fPIC -o mxnet_predict-all.o -c mxnet_predict-all.cc
mxnet_predict-all.cc:31:10: fatal error: 'cuda.h' file not found
#include <cuda.h>
         ^
1 error generated.
make: *** [mxnet_predict-all.o] Error 1

I haven't find a place to turn off CUDA in the Makefile when compiling the Amalgamation. Is it required to have CUDA installed to compile the Amalgamation?

[piiswrong]

@tqchen Could you have a look?

[thirdwing]

I think this should be solved by adding cuda_fp16.h and cuda.h into blacklist in
https://github.com/dmlc/mxnet/blob/master/amalgamation/amalgamation.py#L4

[wqfan]

@thirdwing Hi, Qiang Kou. I followed your instructions by adding those into blacklist. However, I got a new error:

c++ -std=c++11 -Wno-unknown-pragmas -Wall -I/usr/local/opt/openblas/include -shared -o `pwd`/../lib/libmxnet_predict.so mxnet_predict-all.o -L/usr/local/opt/openblas/lib -lopenblas
Undefined symbols for architecture x86_64:
  "mxnet::GraphStorageAllocator::InitStorages()", referenced from:
      mxnet::GraphExecutor::InitDataEntryMemory() in mxnet_predict-all.o
  "mxnet::GraphStorageAllocator::Get(long long, mshadow::TShape)", referenced from:
      mxnet::GraphExecutor::InitDataEntryMemory() in mxnet_predict-all.o
  "mxnet::GraphStorageAllocator::Release(long long, unsigned int)", referenced from:
      mxnet::GraphExecutor::InitDataEntryMemory() in mxnet_predict-all.o
  "mxnet::GraphStorageAllocator::Request(mxnet::Context, int, mshadow::TShape, unsigned int)", referenced from:
      mxnet::GraphExecutor::InitDataEntryMemory() in mxnet_predict-all.o
  "mxnet::GraphStorageAllocator::GraphStorageAllocator(mxnet::StaticGraph*, std::__1::vector<unsigned int, std::__1::allocator<unsigned int> > const&, std::__1::shared_ptr<mxnet::GraphStoragePool>)", referenced from:
      mxnet::GraphExecutor::InitDataEntryMemory() in mxnet_predict-all.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [`pwd`/../lib/libmxnet_predict.so] Error 1

Any suggestions to fix this problem? Thank you.

[ascust]

@thirdwing Hi, I got the same error as @wqfan . It is weird. It thought it was meant to be so easy to get it done.

[thirdwing]

Please have some patience. It works well on Linux.

[JiachenMao]

make clean
make