Deeplearning4j: Compile libnd4j for android

[zhangy10]

Hey guys,

I am trying to compile the libnd4j for android os by the instruction https://github.com/deeplearning4j/deeplearning4j/tree/master/libnd4j#android, but I got the below error:

zzoo@zzoo:~/Documents/nd4j_android/libnd4j$ bash buildnativeoperations.sh -platform android-arm
eval cmake
PACKAGING = none
BUILD = release
CHIP = cpu
ARCH = armv7-a
CHIP_EXTENSION = 
CHIP_VERSION = 
GPU_COMPUTE_CAPABILITY = all
EXPERIMENTAL = no
LIBRARY TYPE = dynamic
OPERATIONS = -DLIBND4J_ALL_OPS=true
MINIFIER = false
NAME = -DLIBND4J_NAME=nd4jcpu
/home/zzoo/Documents/nd4j_android/libnd4j/blasbuild/cpu
-- The C compiler identification is Clang 7.0.2
-- The CXX compiler identification is Clang 7.0.2
-- Check for working C compiler: /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc
-- Check for working C compiler: /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc -- broken
CMake Error at /opt/cmake/share/cmake-3.13/Modules/CMakeTestCCompiler.cmake:52 (message):
  The C compiler

    "/home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/zzoo/Documents/nd4j_android/libnd4j/blasbuild/cpu/CMakeFiles/CMakeTmp

    Run Build Command:"/usr/bin/make" "cmTC_bb71a/fast"
    make -f CMakeFiles/cmTC_bb71a.dir/build.make CMakeFiles/cmTC_bb71a.dir/build
    make[1]: Entering directory '/home/zzoo/Documents/nd4j_android/libnd4j/blasbuild/cpu/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_bb71a.dir/testCCompiler.c.o
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc -DANDROID -D__ANDROID_API__=14 -I/home/zzoo/Android/Sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/include -I/home/zzoo/Android/Sdk/ndk-bundle/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include -I/home/zzoo/Android/Sdk/ndk-bundle/platforms/android-14/arch-arm/usr/include -I/home/zzoo/Android/Sdk/ndk-bundle/sysroot/usr/include -I/home/zzoo/Android/Sdk/ndk-bundle/sysroot/usr/include/arm-linux-androideabi  -fPIC -ffunction-sections -funwind-tables -fstack-protector -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fomit-frame-pointer -fstrict-aliasing -funswitch-loops -finline-limit=300 -Wno-attributes -o CMakeFiles/cmTC_bb71a.dir/testCCompiler.c.o   -c /home/zzoo/Documents/nd4j_android/libnd4j/blasbuild/cpu/CMakeFiles/CMakeTmp/testCCompiler.c
    clang: warning: optimization flag '-funswitch-loops' is not supported [-Wignored-optimization-argument]
    clang: warning: optimization flag '-finline-limit=300' is not supported [-Wignored-optimization-argument]
    Linking C executable cmTC_bb71a
    /opt/cmake/bin/cmake -E cmake_link_script CMakeFiles/cmTC_bb71a.dir/link.txt --verbose=1
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc          CMakeFiles/cmTC_bb71a.dir/testCCompiler.c.o -o cmTC_bb71a  -L/home/zzoo/Android/Sdk/ndk-bundle/platforms/android-14/arch-arm//usr/lib/ --sysroot=/home/zzoo/Android/Sdk/ndk-bundle/platforms/android-14/arch-arm/
    /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
    Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    CMakeFiles/cmTC_bb71a.dir/build.make:86: recipe for target 'cmTC_bb71a' failed
    make[1]: *** [cmTC_bb71a] Error 1
    make[1]: Leaving directory '/home/zzoo/Documents/nd4j_android/libnd4j/blasbuild/cpu/CMakeFiles/CMakeTmp'
    Makefile:121: recipe for target 'cmTC_bb71a/fast' failed
    make: *** [cmTC_bb71a/fast] Error 2




  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)


-- Configuring incomplete, errors occurred!
See also "/home/zzoo/Documents/nd4j_android/libnd4j/blasbuild/cpu/CMakeFiles/CMakeOutput.log".
See also "/home/zzoo/Documents/nd4j_android/libnd4j/blasbuild/cpu/CMakeFiles/CMakeError.log".

I used the latest Cmake 3.13.0, so I am not sure whether the libnd4j is not compatible with the Cmake or I should go with the recommended 3.9 version for the Cmake? So, please any advice for this? Many thanks.

Cheers.

[raver119]

None of your guesses.

  /usr/bin/ld: unrecognised emulation mode: armelf_linux_eabi
    Supported emulations: elf_x86_64 elf32_x86_64 elf_i386 elf_iamcu i386linux elf_l1om elf_k1om i386pep i386pe
    clang: error: linker command failed with exit code 1 (use -v to see invocation)

Linker says it doesn't know how to link to arm.

[zhangy10]

@raver119 Thanks for that and please any solutions for this? And I used Nexus 5x with arm64 architecture.

[saudet]

@zhangy10 It looks like you're trying to build an old version of libnd4j, can you try with the latest code from the master branch?

[zhangy10]

@saudet I thought I just followed with the instructions of master branch:

git clone https://github.com/deeplearning4j/libnd4j
git clone https://github.com/deeplearning4j/nd4j
export ANDROID_NDK=/path/to/android-ndk/
cd libnd4j
bash buildnativeoperations.sh -platform android-xxx
cd ../nd4j
mvn clean install -Djavacpp.platform=android-xxx -DskipTests -pl '!:nd4j-cuda-9.0,!:nd4j-cuda-9.0-platform,!:nd4j-tests'

So, the link https://github.com/deeplearning4j/libnd4j isn't the master branch?

[saudet]

Ah, we need to update the README.md file.

Use the master branch of the directory where the README.md file is in.

[raver119]

@zhangy10 yes, it's not master. Please note repository where you've created this issue, and repository you're trying to build.

[raver119]

p.s.

git clone https://github.com/deeplearning4j/deeplearning4j.git
cd deeplearning4j/libnd4j

[zhangy10]

@saudet Thanks so much and I will have a try for that!

[zhangy10]

@saudet Just tried the master branch but still got this error:

zzoo@zzoo:~/Documents/nd4j_android/deeplearning4j/libnd4j$ bash buildnativeoperations.sh -platform android-arm
eval cmake
Could not find MKL-DNN, please make sure to run the build with Maven
Could not find OpenBLAS, please make sure to run the build with Maven
PACKAGING = none
BUILD = release
CHIP = cpu
ARCH = armv7-a
CHIP_EXTENSION = 
CHIP_VERSION = 
GPU_COMPUTE_CAPABILITY = all
EXPERIMENTAL = no
LIBRARY TYPE = dynamic
OPERATIONS = -DLIBND4J_ALL_OPS=true
MINIFIER = -DLIBND4J_BUILD_MINIFIER=false
TESTS = -DBUILD_TESTS=OFF
NAME = -DLIBND4J_NAME=nd4jcpu
MKLDNN_PATH = 
OPENBLAS_PATH = 
/home/zzoo/Documents/nd4j_android/deeplearning4j/libnd4j/blasbuild/cpu
-- The C compiler identification is Clang 7.0.2
-- The CXX compiler identification is Clang 7.0.2
-- Check for working C compiler: /home/zzoo/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64//bin/clang
-- Check for working C compiler: /home/zzoo/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64//bin/clang -- broken
CMake Error at /opt/cmake/share/cmake-3.13/Modules/CMakeTestCCompiler.cmake:52 (message):
  The C compiler

    "/home/zzoo/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64//bin/clang"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /home/zzoo/Documents/nd4j_android/deeplearning4j/libnd4j/blasbuild/cpu/CMakeFiles/CMakeTmp

    Run Build Command:"/usr/bin/make" "cmTC_39ac5/fast"
    make -f CMakeFiles/cmTC_39ac5.dir/build.make CMakeFiles/cmTC_39ac5.dir/build
    make[1]: Entering directory '/home/zzoo/Documents/nd4j_android/deeplearning4j/libnd4j/blasbuild/cpu/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_39ac5.dir/testCCompiler.c.o
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64//bin/clang -DANDROID -D__ANDROID_API__=14 -I/home/zzoo/Android/Sdk/ndk-bundle/sources/cxx-stl/llvm-libc++/include -I/home/zzoo/Android/Sdk/ndk-bundle/sources/cxx-stl/llvm-libc++/../llvm-libc++abi/include -I/home/zzoo/Android/Sdk/ndk-bundle/sources/android/support/include -I/home/zzoo/Android/Sdk/ndk-bundle/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a/include -I/home/zzoo/Android/Sdk/ndk-bundle/sysroot/usr/include -I/home/zzoo/Android/Sdk/ndk-bundle/sysroot/usr/include/arm-linux-androideabi -I/home/zzoo/Android/Sdk/ndk-bundle/platforms/android-14/arch-arm/usr/include  -fPIC -ffunction-sections -funwind-tables -fstack-protector-strong -target armv7-none-linux-androideabi -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -o CMakeFiles/cmTC_39ac5.dir/testCCompiler.c.o   -c /home/zzoo/Documents/nd4j_android/deeplearning4j/libnd4j/blasbuild/cpu/CMakeFiles/CMakeTmp/testCCompiler.c
    Linking C executable cmTC_39ac5
    /opt/cmake/bin/cmake -E cmake_link_script CMakeFiles/cmTC_39ac5.dir/link.txt --verbose=1
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64//bin/clang          CMakeFiles/cmTC_39ac5.dir/testCCompiler.c.o -target armv7-none-linux-androideabi14 -Wl,--fix-cortex-a8 -Wl,--no-undefined -z text -o cmTC_39ac5  -gcc-toolchain /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/ --sysroot=/home/zzoo/Android/Sdk/ndk-bundle/platforms/android-14/arch-arm/ -lc -lm
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot open crtbegin_dynamic.o: No such file or directory
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot open crtend_android.o: No such file or directory
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lc
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lm
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -ldl
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -lc
    /home/zzoo/Android/Sdk/ndk-bundle/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld: error: cannot find -ldl
    /buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/unwind-arm-common.inc:289: error: undefined reference to 'abort'
    /buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/unwind-arm-common.inc:346: error: undefined reference to 'memcpy'
    /buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/unwind-arm-common.inc:376: error: undefined reference to 'memcpy'
    /buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/unwind-arm-common.inc:505: error: undefined reference to 'abort'
    /buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/config/arm/pr-support.c:378: error: undefined reference to 'abort'
    /buildbot/src/android/gcc/toolchain/build/../gcc/gcc-4.9/libgcc/config/arm/pr-support.c:384: error: undefined reference to 'abort'
    clang: error: linker command failed with exit code 1 (use -v to see invocation)
    CMakeFiles/cmTC_39ac5.dir/build.make:86: recipe for target 'cmTC_39ac5' failed
    make[1]: *** [cmTC_39ac5] Error 1
    make[1]: Leaving directory '/home/zzoo/Documents/nd4j_android/deeplearning4j/libnd4j/blasbuild/cpu/CMakeFiles/CMakeTmp'
    Makefile:121: recipe for target 'cmTC_39ac5/fast' failed
    make: *** [cmTC_39ac5/fast] Error 2


  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)


-- Configuring incomplete, errors occurred!
See also "/home/zzoo/Documents/nd4j_android/deeplearning4j/libnd4j/blasbuild/cpu/CMakeFiles/CMakeOutput.log".
See also "/home/zzoo/Documents/nd4j_android/deeplearning4j/libnd4j/blasbuild/cpu/CMakeFiles/CMakeError.log".

And as the compiling process is a bit complex, I think I am a bit losing the track. Actually, I followed with this link https://deeplearning4j.org/docs/latest/deeplearning4j-build-from-source to build the project. I first set up some essential prerequisites like maven, OpenMPL, Cmake, OpenBLAS, ATLAS, etc. And I skipped setting the cuda. During the checking ENV part, I thought I should first build libnd4j and add it to the PATH but I was stuck here.

So, am I on the right track if this is for Android platform? Or I should just simply go with these scripts mentioned in the master branch:

./change-cuda-versions.sh x.x
./change-scala-versions.sh 2.xx
./change-spark-versions.sh x
mvn clean install -Dmaven.test.skip -Dlibnd4j.cuda=x.x -Dlibnd4j.compute=xx

If so, what does the x.x or 2.xx mean?

Thanks.

[saudet]

Which version of the NDK are you trying to use?

[zhangy10]

@saudet I think it's the latest 18.1.5063045, and using the Android studio to download it. So, is it compatible?

[zhangy10]

@saudet And I saw these outputs:

Could not find MKL-DNN, please make sure to run the build with Maven
Could not find OpenBLAS, please make sure to run the build with Maven

I think I forgot to set the openblas into PATH and I will do that later. And do I also need to install MKL?

[saudet]

Maybe not compatible no, let me check it out...

[zhangy10]

@saudet Thanks and if not let me know which NDK version is good for compiling.

[saudet]

r15c and r16b have been working well, something else to add to the documentation :)

[zhangy10]

@saudet Great thanks for that!

And can I double check the whole steps of compiling?

1. Setup prerequisites: maven, OpenMPL, Cmake, OpenBLAS, ATLAS, (skipped Cuda) and MKL(not yet install)

1. Check ENV
2. build libnd4j
3. build nd4j

4. build datavec? (not sure it's required)

5. Run the script to build dl4j

./buildmultiplescalaversions.sh clean install -DskipTests -Dmaven.javadoc.skip=true

1. Once all good, set repositories {mavenLocal()} to use in the Android Proj

Are they all good?

[saudet]

For Android, you're not going to need anything else than the NDK!

[saudet]

To build everything including the Java modules, execute the build with mvn -Djavacpp.platform=android-arm ... instead of running the script manually.

[saudet]

And finally, instructions to configure a Gradle build are available here:
https://deeplearning4j.org/docs/latest/deeplearning4j-android

[zhangy10]

@saudet Great! Thanks so much and hope it works well.

[zhangy10]

@saudet Just got another error when compiling:

[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireActiveProfile failed with message:
Profile "testresources" is not activated.

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] libnd4j ............................................ SUCCESS [20:56 min]
[INFO] nd4j ............................................... FAILURE [16:25 min]
[INFO] nd4j-shade ......................................... SKIPPED
[INFO] jackson ............................................ SKIPPED
[INFO] nd4j-common ........................................ SKIPPED
[INFO] nd4j-context ....................................... SKIPPED
[INFO] nd4j-buffer ........................................ SKIPPED
[INFO] nd4j-backends ...................................... SKIPPED
[INFO] nd4j-api-parent .................................... SKIPPED
[INFO] nd4j-api ........................................... SKIPPED
[INFO] nd4j-jdbc .......................................... SKIPPED
[INFO] nd4j-jdbc-api ...................................... SKIPPED
[INFO] nd4j-jdbc-mysql .................................... SKIPPED
[INFO] nd4j-jdbc-hsql ..................................... SKIPPED
[INFO] nd4j-instrumentation ............................... SKIPPED
[INFO] nd4j-serde ......................................... SKIPPED
[INFO] nd4j-aeron ......................................... SKIPPED
[INFO] nd4j-jackson ....................................... SKIPPED
[INFO] nd4j-kryo .......................................... SKIPPED
[INFO] nd4j-camel-routes .................................. SKIPPED
[INFO] nd4j-kafka ......................................... SKIPPED
[INFO] nd4j-gson .......................................... SKIPPED
[INFO] nd4j-arrow ......................................... SKIPPED
[INFO] nd4j-backend-impls ................................. SKIPPED
[INFO] nd4j-native-api .................................... SKIPPED
[INFO] nd4j-tensorflow .................................... SKIPPED
[INFO] nd4j-native ........................................ SKIPPED
[INFO] nd4j-grpc .......................................... SKIPPED
[INFO] nd4j-tests ......................................... SKIPPED
[INFO] nd4j-native-platform ............................... SKIPPED
[INFO] nd4j-cuda .......................................... SKIPPED
[INFO] nd4j-cuda-platform ................................. SKIPPED
[INFO] nd4j-tests-tensorflow .............................. SKIPPED
[INFO] nd4j-parameter-server-parent ....................... SKIPPED
[INFO] nd4j-parameter-server-model ........................ SKIPPED
[INFO] nd4j-parameter-server .............................. SKIPPED
[INFO] nd4j-parameter-server-status ....................... SKIPPED
[INFO] nd4j-parameter-server-client ....................... SKIPPED
[INFO] nd4j-parameter-server-rocksdb-storage .............. SKIPPED
[INFO] nd4j-parameter-server-node ......................... SKIPPED
[INFO] nd4j-uberjar ....................................... SKIPPED
[INFO] DataVec ............................................ SKIPPED
[INFO] datavec-api ........................................ SKIPPED
[INFO] datavec-data ....................................... SKIPPED
[INFO] datavec-data-audio ................................. SKIPPED
[INFO] datavec-data-image ................................. SKIPPED
[INFO] datavec-data-codec ................................. SKIPPED
[INFO] datavec-data-nlp ................................... SKIPPED
[INFO] datavec-geo ........................................ SKIPPED
[INFO] datavec-hadoop ..................................... SKIPPED
[INFO] datavec-arrow ...................................... SKIPPED
[INFO] datavec-local ...................................... SKIPPED
[INFO] datavec-spark_2.11 ................................. SKIPPED
[INFO] DataVec Camel Component ............................ SKIPPED
[INFO] datavec-spark-inference-parent ..................... SKIPPED
[INFO] datavec-spark-inference-model ...................... SKIPPED
[INFO] datavec-spark-inference-server ..................... SKIPPED
[INFO] datavec-spark-inference-client ..................... SKIPPED
[INFO] datavec-jdbc ....................................... SKIPPED
[INFO] datavec-excel ...................................... SKIPPED
[INFO] datavec-perf ....................................... SKIPPED
[INFO] DeepLearning4j ..................................... SKIPPED
[INFO] deeplearning4j-util ................................ SKIPPED
[INFO] deeplearning4j-data ................................ SKIPPED
[INFO] deeplearning4j-utility-iterators ................... SKIPPED
[INFO] deeplearning4j-common .............................. SKIPPED
[INFO] deeplearning4j-nn .................................. SKIPPED
[INFO] deeplearning4j-datavec-iterators ................... SKIPPED
[INFO] deeplearning4j-datasets ............................ SKIPPED
[INFO] deeplearning4j-nearestneighbors-parent ............. SKIPPED
[INFO] nearestneighbor-core ............................... SKIPPED
[INFO] deeplearning4j-manifold ............................ SKIPPED
[INFO] deeplearning4j-tsne ................................ SKIPPED
[INFO] deeplearning4j-modelimport ......................... SKIPPED
[INFO] deeplearning4j-ui-parent ........................... SKIPPED
[INFO] deeplearning4j-ui-components ....................... SKIPPED
[INFO] deeplearning4j-core ................................ SKIPPED
[INFO] DeepLearning4j-scaleout-parent ..................... SKIPPED
[INFO] DeepLearning4j-AWS ................................. SKIPPED
[INFO] deeplearning4j-ui-model ............................ SKIPPED
[INFO] deeplearning4j-play ................................ SKIPPED
[INFO] Spark parent ....................................... SKIPPED
[INFO] dl4j-spark ......................................... SKIPPED
[INFO] deeplearning4j-nlp-parent .......................... SKIPPED
[INFO] deeplearning4j-nlp ................................. SKIPPED
[INFO] dl4j-spark-nlp ..................................... SKIPPED
[INFO] deeplearning4j-parallel-wrapper .................... SKIPPED
[INFO] dl4j-spark-parameterserver ......................... SKIPPED
[INFO] dl4j-spark-nlp-java8 ............................... SKIPPED
[INFO] dl4j-spark-ml ...................................... SKIPPED
[INFO] deeplearning4j-scaleout-parallelwrapper-parameter-server SKIPPED
[INFO] deeplearning4j-ui-resources ........................ SKIPPED
[INFO] deeplearning4j-ui .................................. SKIPPED
[INFO] deeplearning4j-ui-standalone ....................... SKIPPED
[INFO] deeplearning4j-graph ............................... SKIPPED
[INFO] deeplearning4j-cuda ................................ SKIPPED
[INFO] deeplearning4j-nlp-uima ............................ SKIPPED
[INFO] deeplearning4j-nlp-korean_2.11 ..................... SKIPPED
[INFO] deeplearning4j-nlp-japanese ........................ SKIPPED
[INFO] deeplearning4j-nlp-chinese ......................... SKIPPED
[INFO] deeplearning4j-dataimport-solrj .................... SKIPPED
[INFO] deeplearning4j-modelexport-solr .................... SKIPPED
[INFO] deeplearning4j-zoo ................................. SKIPPED
[INFO] deeplearning4j-nearestneighbors-model .............. SKIPPED
[INFO] deeplearning4j-nearestneighbors-client ............. SKIPPED
[INFO] deeplearning4j-nearestneighbor-server .............. SKIPPED
[INFO] dl4j-perf .......................................... SKIPPED
[INFO] dl4j-integration-tests ............................. SKIPPED
[INFO] Arbiter ............................................ SKIPPED
[INFO] arbiter-core ....................................... SKIPPED
[INFO] arbiter-deeplearning4j ............................. SKIPPED
[INFO] arbiter-server ..................................... SKIPPED
[INFO] arbiter-ui ......................................... SKIPPED
[INFO] gym-java-client .................................... SKIPPED
[INFO] rl4j ............................................... SKIPPED
[INFO] rl4j-api ........................................... SKIPPED
[INFO] rl4j-core .......................................... SKIPPED
[INFO] rl4j-gym ........................................... SKIPPED
[INFO] rl4j-doom .......................................... SKIPPED
[INFO] rl4j-ale ........................................... SKIPPED
[INFO] rl4j-malmo ......................................... SKIPPED
[INFO] ScalNet ............................................ SKIPPED
[INFO] jumpy .............................................. SKIPPED
[INFO] pydatavec .......................................... SKIPPED
[INFO] pydl4j ............................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 37:24 min
[INFO] Finished at: 2018-11-28T13:18:48+11:00
[INFO] Final Memory: 50M/445M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-test-resources) on project nd4j: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :nd4j

And it says 'Profile "testresources" is not activated' for the nd4j, so how to activate the test resources for that? Or just simply type the command: mvn clean install -Dtests.skip=true -rf :nd4j to skip the test resources?

[AlexDBlack]

@zhangy10 Yes, just skip tests (and skip javadoc for faster build, unless you need it)
mvn clean install -Dmaven.test.skip -Dmaven.javadoc.skip=true
If you're not building for CUDA (not sure if you are) you'll want to skip the nd4j-cuda etc modules too.

[zhangy10]

@AlexDBlack Thanks so much and it works!

[zhangy10]

@AlexDBlack Can I ask how to skip the nd4j-cuda modules as just got another error of building the nd4j-cuda?

[WARNING] Rule 1: org.apache.maven.plugins.enforcer.RequireFilesExist failed with message:
!!! You have to compile libnd4j with cuda support first!
Some required files are missing:
/home/zzoo/Documents/dl4j_android/deeplearning4j/nd4j/nd4j-backends/nd4j-backend-impls/nd4j-cuda/../../../../libnd4j/blasbuild/cuda/blas

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] nd4j ............................................... SUCCESS [09:15 min]
[INFO] nd4j-shade ......................................... SUCCESS [  0.247 s]
[INFO] jackson ............................................ SUCCESS [24:38 min]
[INFO] nd4j-common ........................................ SUCCESS [02:15 min]
[INFO] nd4j-context ....................................... SUCCESS [  1.584 s]
[INFO] nd4j-buffer ........................................ SUCCESS [  1.893 s]
[INFO] nd4j-backends ...................................... SUCCESS [  0.138 s]
[INFO] nd4j-api-parent .................................... SUCCESS [  0.148 s]
[INFO] nd4j-api ........................................... SUCCESS [02:54 min]
[INFO] nd4j-jdbc .......................................... SUCCESS [  0.167 s]
[INFO] nd4j-jdbc-api ...................................... SUCCESS [ 37.987 s]
[INFO] nd4j-jdbc-mysql .................................... SUCCESS [  1.750 s]
[INFO] nd4j-jdbc-hsql ..................................... SUCCESS [ 24.515 s]
[INFO] nd4j-instrumentation ............................... SUCCESS [10:28 min]
[INFO] nd4j-serde ......................................... SUCCESS [  0.111 s]
[INFO] nd4j-aeron ......................................... SUCCESS [ 25.464 s]
[INFO] nd4j-jackson ....................................... SUCCESS [  1.303 s]
[INFO] nd4j-kryo .......................................... SUCCESS [14:23 min]
[INFO] nd4j-camel-routes .................................. SUCCESS [  0.141 s]
[INFO] nd4j-kafka ......................................... SUCCESS [03:21 min]
[INFO] nd4j-gson .......................................... SUCCESS [ 26.593 s]
[INFO] nd4j-arrow ......................................... SUCCESS [03:36 min]
[INFO] nd4j-backend-impls ................................. SUCCESS [  0.164 s]
[INFO] nd4j-native-api .................................... SUCCESS [  1.725 s]
[INFO] nd4j-tensorflow .................................... SUCCESS [ 19.649 s]
[INFO] nd4j-native ........................................ SUCCESS [05:44 min]
[INFO] nd4j-grpc .......................................... SUCCESS [06:39 min]
[INFO] nd4j-tests ......................................... SUCCESS [ 27.436 s]
[INFO] nd4j-native-platform ............................... SUCCESS [ 52.959 s]
[INFO] nd4j-cuda .......................................... FAILURE [ 41.332 s]
[INFO] nd4j-cuda-platform ................................. SKIPPED

And I thought I could use this mvn clean install -Djavacpp.platform=android-arm -DskipTests -Dmaven.javadoc.skip=true -pl '!:nd4j-cuda-10.0,!:nd4j-cuda-10.0-platform,!:nd4j-tests' -rf :nd4j-cuda-10.0 to skip the cuda and resume the building but it doesn't work.

[AlexDBlack]

@zhangy10 looks like you're close with that command. Maybe try this:
mvn clean install -pl '!deeplearning4j/deeplearning4j-cuda,!nd4j/nd4j-backends/nd4j-backend-impls/nd4j-cuda,!nd4j/nd4j-backends/nd4j-backend-impls/nd4j-cuda-platform' -Dlibnd4j.platform=android-arm -Djavacpp.platform=android-arm -Dmaven.test.skip=true -Dmaven.javadoc.skip=true

[zhangy10]

@AlexDBlack Thanks mate and I will have a try. Just wondering what if I modify a few parts of codes could I just re-build the modified sub-modules instead of the full stacks? Like if I just make a small change for the deeplearning4j-core module after the full compiling, could I just re-build this single one to save time?

[AlexDBlack]

@zhangy10 yes, if you modify just one module, you only need to rebuild that module... unless there's API changes, then obviously you also need to update and rebuild any downstream modules.

You might be able to get away with using snapshots for the other modules too (it should auto download them if you haven't built them recently?)

[zhangy10]

@AlexDBlack Thanks so much for that and finally building success! And I will continue to check the submodule rebuild stuff. Anyways, big thanks again.

[zhangy10]

@AlexDBlack Just got another question so what if I would like to build both android-arm and android-arm64 versions for nd4j, how should I put the command? Or like this -Dlibnd4j.platform="android-arm android-arm64" using the quotation and space marks for multi-argument? Actually, one of my phones is arm64 based and I think I just build the android-arm version. So, should I clean everything and rebuild both versions at the same time? Many thanks.

[saudet]

We can't build more than one platform at a time. You'll have to launch the build for libnd4j and nd4j multiple times.

[zhangy10]

@saudet All right, I see. Thanks for that!

[zhangy10]

@saudet @AlexDBlack I launched the build 3 times for android-arm, android-arm64, android-x86_64 and all build are good. However, when I set up the gradle file and run the project, I got these errors:

More than one file was found with OS independent path: 'lib/armeabi-v7a/libopenblas.so'
More than one file was found with OS independent path: 'lib/arm64-v8a/libopenblas.so'
More than one file was found with OS independent path: 'lib/x86_64/libopenblas.so'

And it seems that there are 3 copies of libopenblas.so in the dependency libraries. Then, I got one solution that I put these in the gradle file:

packagingOptions {
            exclude 'META-INF/DEPENDENCIES'
            exclude 'META-INF/DEPENDENCIES.txt'
            exclude 'META-INF/LICENSE'
            exclude 'META-INF/LICENSE.txt'
            exclude 'META-INF/license.txt'
            exclude 'META-INF/NOTICE'
            exclude 'META-INF/NOTICE.txt'
            exclude 'META-INF/notice.txt'
            exclude 'META-INF/INDEX.LIST'

            // The solution for multiple libopenblas.so
           pickFirst 'lib/armeabi-v7a/libopenblas.so'
           pickFirst 'lib/arm64-v8a/libopenblas.so'
           pickFirst 'lib/x86_64/libopenblas.so'
}

It works for the compiling the android project and I can get the apk, but I got another error when I install the apk to phone and run it:

Caused by: java.lang.UnsatisfiedLinkError: dlopen failed: "/data/app/org.deeplearning4j.examples.test/lib/arm64/libopenblas.so" has unexpected e_machine: 62
12-04 13:50:32.465 11779 12055 E AndroidRuntime:    at java.lang.Runtime.loadLibrary(Runtime.java:372)
12-04 13:50:32.465 11779 12055 E AndroidRuntime:    at java.lang.System.loadLibrary(System.java:1076)
12-04 13:50:32.465 11779 12055 E AndroidRuntime:    at org.bytedeco.javacpp.Loader.loadLibrary(Loader.java:1258)
12-04 13:50:32.465 11779 12055 E AndroidRuntime:    at org.bytedeco.javacpp.Loader.load(Loader.java:977)
12-04 13:50:32.465 11779 12055 E AndroidRuntime:    ... 25 more

It seems that the arm64/libopenblas.so is missing or skipped during the compiling based on the above solution. So, I think the pickFirst can't work to exclude the multiple libopenblas.so.

So, can I ask how to deal with the multiple libopenblas.so issue? Many thanks again.

[saudet]

It just looks like your Gradle build is picking up more than one version of the binaries for OpenBLAS. Try to figure out where they are coming from and remove the redundant artifacts.

[zhangy10]

@saudet All right, thanks and I'll double check that.

[zhangy10]

@saudet I found the redundant artifacts and removed it, but I still got this issue after installing the apk:

Caused by: java.lang.UnsatisfiedLinkError: dlopen failed: "/data/app/org.deeplearning4j.examples.test/lib/arm64/libopenblas.so" has unexpected e_machine:

And some people said that it could be a problem of the NDK version. As we discussed, here I used r16b for the build. So, can I double check which version of NDK are you using for building the released 1.0.0-beta3? Actually, I tested the beta3 and it worked perfectly on my phones.

[saudet]

We use r16b for releases right now, and yes it works fine.

[saudet]

BTW, if you just need to get something fixed, it would be easier to send us a pull request, and get snapshot builds that way.

[zhangy10]

@saudet Thanks for letting me know that and I'll double check again. If still not fixed, I will send a pull request.

[zhangy10]

@saudet I just found that the multiple 'lib.arm64-v8a.libopenblas.so' were coming from both 'org.bytedeco.javacpp-presets.openblas' (0.3.3-1.4.3) and 'org.nd4j.nd4j-native' (rebuild 1.0.0-SNAPSHOT). And I think I removed the second one then I got the above 'failed open the libopenblas.so' issue, but not sure removing the first one works.

So, I am just wondering why there are 2 copies of the libopenblas.so in the dl4j libs and which one should really exclude?

[saudet]

No clue, doesn't do that either here, in the snapshots, or in releases. Works fine with this build command:

mvn  install -Dmaven.test.skip -Djavacpp.platform=android-arm64 -pl '!:nd4j-cuda-10.0,!:nd4j-cuda-10.0-platform,!:deeplearning4j-cuda-10.0'

[zhangy10]

@saudet Thanks mate! I will have a try again to figure it out.