Bazel: Windows: build TensorFlow Lite with Bazel on Windows

Created on 22 Nov 2017 · 15Comments · Source: bazelbuild/bazel

This is a tracking bug for building TF Lite on Windows with Bazel.

[x] remove -Wno-implicit-fallthrough from @tensorflow//third_party/flatbuffers/flatbuffers.BUILD
[x] remove single quotes from repro command's --cxxflags
[ ] update README.md: use drive root as --output_user_root (long paths)
[ ] update README.md: build with "-c opt" (long paths)
[ ] edit WindowsJniLoader to load from bazel_tools, not from <reponame>/external/bazel_tools (must test on Android BusyBox)
[ ] create src/main/native/windows/BUILD.tools
[ ] add src/main/native/windows/*.{cc,h} to //src/main/native/windows:embedded_tools
[x] Java SingleJar doesn't support --nocompress_suffixes, and some build rule requires that --> migrate to C++ SingleJar on Windows (#2241)

P2 area-Windows windows team-XProduct feature request

Source

laszlocsomor

All 15 comments

/cc: @meteorcloudy

laszlocsomor on 30 Nov 2017

Raising to P1. I want to do this in Q1-2018.

laszlocsomor on 13 Dec 2017

Update: I'm not actively working on this, unfortunately. I'm focused on #4460, #4292, and #4319. Maybe P1 is appropriate but I cannot seem to deliver that. @meteorcloudy , @dslomov : WDYT about dropping the priority? If you think it's still P1, how do you think we could find time for this?

laszlocsomor on 16 May 2018

I think we can drop the priority to P2. I talked to people working on TensorFlow lite, they don't seem to need this urgently. We can reconsider the priority after TF completely moves to Bazel on Windows, which is what I'm working on right now.

meteorcloudy on 16 May 2018

👎8 👍1

I tried building TFLite at https://github.com/tensorflow/tensorflow/commit/3e21fe5faedab3a8258d344c8ad1cec2612a8aa8, using Bazel 0.22.0 (configure.py complains if you use 0.23.0), with Python3 on my PATH, on Windows 10.

It fails with both MSVC and with GCC, but it gets pretty far:

With MSVC:

C:\src\tensorflow\tensorflow\lite>bazel build examples/...
...
ERROR: C:/src/tensorflow/tensorflow/lite/examples/label_image/BUILD:37:1: C++ compilation of rule '//tensorflow/lite/examples/label_image:bitmap_helpers' failed (Exit 2): cl.exe failed: error executing command
  cd C:/_bazel/qukqnr66/execroot/org_tensorflow
  SET INCLUDE=C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\include;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.6.1\include\um;C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\ucrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\shared;C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\um;C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\winrt;C:\Program Files (x86)\Windows Kits\10\include\10.0.17763.0\cppwinrt
    SET PATH=C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64;C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\MSBuild\15.0\bin\Roslyn;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6.1 Tools\x64\;C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\x64;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\\MSBuild\15.0\bin;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\;C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\Tools\;;C:\Windows\system32;C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja
    SET PWD=/proc/self/cwd
    SET PYTHON_BIN_PATH=c:/Python37/python.exe
    SET PYTHON_LIB_PATH=c:/Python37/lib/site-packages
    SET TEMP=c:\tmp
    SET TF_DOWNLOAD_CLANG=0
    SET TF_NEED_CUDA=0
    SET TF_NEED_OPENCL_SYCL=0
    SET TF_NEED_ROCM=0
    SET TMP=c:\tmp
  C:/Program Files (x86)/Microsoft Visual Studio/2017/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX64/x64/cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-opt/genfiles /Ibazel-out/x64_windows-opt/bin /Iexternal/flatbuffers /Ibazel-out/x64_windows-opt/genfiles/external/flatbuffers /Ibazel-out/x64_windows-opt/bin/external/flatbuffers /Iexternal/gemmlowp /Ibazel-out/x64_windows-opt/genfiles/external/gemmlowp /Ibazel-out/x64_windows-opt/bin/external/gemmlowp /Iexternal/com_google_absl /Ibazel-out/x64_windows-opt/genfiles/external/com_google_absl /Ibazel-out/x64_windows-opt/bin/external/com_google_absl /Iexternal/arm_neon_2_x86_sse /Ibazel-out/x64_windows-opt/genfiles/external/arm_neon_2_x86_sse /Ibazel-out/x64_windows-opt/bin/external/arm_neon_2_x86_sse /Iexternal/eigen_archive /Ibazel-out/x64_windows-opt/genfiles/external/eigen_archive /Ibazel-out/x64_windows-opt/bin/external/eigen_archive /Iexternal/local_config_sycl /Ibazel-out/x64_windows-opt/genfiles/external/local_config_sycl /Ibazel-out/x64_windows-opt/bin/external/local_config_sycl /Iexternal/fft2d /Ibazel-out/x64_windows-opt/genfiles/external/fft2d /Ibazel-out/x64_windows-opt/bin/external/fft2d /Iexternal/farmhash_archive /Ibazel-out/x64_windows-opt/genfiles/external/farmhash_archive /Ibazel-out/x64_windows-opt/bin/external/farmhash_archive /Itensorflow/lite/schema /Ibazel-out/x64_windows-opt/genfiles/tensorflow/lite/schema /Ibazel-out/x64_windows-opt/bin/tensorflow/lite/schema /Iexternal/flatbuffers/include /Ibazel-out/x64_windows-opt/genfiles/external/flatbuffers/include /Ibazel-out/x64_windows-opt/bin/external/flatbuffers/include /Iexternal/eigen_archive /Ibazel-out/x64_windows-opt/genfiles/external/eigen_archive /Ibazel-out/x64_windows-opt/bin/external/eigen_archive /Iexternal/farmhash_archive/src /Ibazel-out/x64_windows-opt/genfiles/external/farmhash_archive/src /Ibazel-out/x64_windows-opt/bin/external/farmhash_archive/src /DEIGEN_MPL2_ONLY /DEIGEN_MAX_ALIGN_BYTES=64 /DEIGEN_HAS_TYPE_TRAITS=0 /showIncludes /MD /O2 /Oy- /DNDEBUG /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Gy /Gw -w /Fobazel-out/x64_windows-opt/bin/tensorflow/lite/examples/label_image/_objs/bitmap_helpers/bitmap_helpers.obj /c tensorflow/lite/examples/label_image/bitmap_helpers.cc
Execution platform: @bazel_tools//platforms:host_platform
tensorflow/lite/examples/label_image/bitmap_helpers.cc(22): fatal error C1083: Cannot open include file: 'unistd.h': No such file or directory
INFO: Elapsed time: 177.311s, Critical Path: 28.41s
INFO: 119 processes: 119 local.
FAILED: Build did NOT complete successfully

Apparently something tries to include unistd.h which doesn't exist in MSVC.

With GCC:

C:\src\tensorflow\tensorflow\lite>bazel build examples/... --cpu=x64_windows --compiler=msys-gcc
...
ERROR: C:/_bazel/qukqnr66/external/com_google_absl/absl/types/BUILD.bazel:163:1: C++ compilation of rule '@com_google_absl//absl/types:optional' failed (Exit 1): gcc failed: error executing command
  cd C:/_bazel/qukqnr66/execroot/org_tensorflow
  SET PATH=c:/msys64/usr/bin
    SET PWD=/proc/self/cwd
    SET PYTHON_BIN_PATH=c:/Python37/python.exe
    SET PYTHON_LIB_PATH=c:/Python37/lib/site-packages
    SET TF_DOWNLOAD_CLANG=0
    SET TF_NEED_CUDA=0
    SET TF_NEED_OPENCL_SYCL=0
    SET TF_NEED_ROCM=0
  c:/msys64/usr/bin/gcc -std=gnu++0x -MD -MF bazel-out/x64_windows-opt/bin/external/com_google_absl/absl/types/_objs/optional/optional.d -frandom-seed=bazel-out/x64_windows-opt/bin/external/com_google_absl/absl/types/_objs/optional/optional.o -D__CLANG_SUPPORT_DYN_ANNOTATION__ -iquote external/com_google_absl -iquote bazel-out/x64_windows-opt/genfiles/external/com_google_absl -iquote bazel-out/x64_windows-opt/bin/external/com_google_absl -w /W3 /wd4005 /wd4068 /wd4180 /wd4244 /wd4267 /wd4800 /DNOMINMAX /DWIN32_LEAN_AND_MEAN /D_CRT_SECURE_NO_WARNINGS /D_SCL_SECURE_NO_WARNINGS /D_ENABLE_EXTENDED_ALIGNED_STORAGE -c external/com_google_absl/absl/types/optional.cc -o bazel-out/x64_windows-opt/bin/external/com_google_absl/absl/types/_objs/optional/optional.o
Execution platform: @bazel_tools//platforms:host_platform
gcc: error: /W3: No such file or directory
gcc: error: /wd4005: No such file or directory
gcc: error: /wd4068: No such file or directory
gcc: error: /wd4180: No such file or directory
gcc: error: /wd4244: No such file or directory
gcc: error: /wd4267: No such file or directory
gcc: error: /wd4800: No such file or directory
gcc: error: /DNOMINMAX: No such file or directory
gcc: error: /DWIN32_LEAN_AND_MEAN: No such file or directory
gcc: error: /D_CRT_SECURE_NO_WARNINGS: No such file or directory
gcc: error: /D_SCL_SECURE_NO_WARNINGS: No such file or directory
gcc: error: /D_ENABLE_EXTENDED_ALIGNED_STORAGE: No such file or directory
INFO: Elapsed time: 49.502s, Critical Path: 10.98s
INFO: 166 processes: 166 local.
FAILED: Build did NOT complete successfully

laszlocsomor on 28 Feb 2019

https://www.tensorflow.org/lite/demo_android#build_tensorflow_lite_and_the_demo_app_from_source
Building the Android Demo should work on Windows

bazel build --cxxopt=--std=c++11 //tensorflow/lite/java/demo/app/src/main:TfLiteCameraDemo

meteorcloudy on 28 Feb 2019

The Android Demo consistently crashes for me (both with Bazel 0.22.0 and 0.23.0):

C:\src\tensorflow>bazel build --cxxopt=--std=c++11 //tensorflow/lite/java/demo/app/src/main:TfLiteCameraDemo

...

ERROR: C:/_bazel/qukqnr66/external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/BUILD:8:1: Building external/bazel_tools/src/tools/android/java/com/google/devtools/build/android/ResourceProcessorBusyBox.jar () failed: Worker process quit or closed its stdin stream when we tried to send a WorkRequest:

---8<---8<--- Exception details ---8<---8<---
java.io.IOException: ERROR: src/main/native/windows/processes-jni.cc(594): NativeProcess:WriteStdin(17664): The pipe is being closed.

        at com.google.devtools.build.lib.windows.WindowsSubprocess.writeStream(WindowsSubprocess.java:263)
        at com.google.devtools.build.lib.windows.WindowsSubprocess.access$000(WindowsSubprocess.java:33)
        at com.google.devtools.build.lib.windows.WindowsSubprocess$ProcessOutputStream.write(WindowsSubprocess.java:57)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.doFlush(CodedOutputStream.java:3003)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.write(CodedOutputStream.java:2935)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeLazy(CodedOutputStream.java:2954)
        at com.google.protobuf.ByteString$LiteralByteString.writeTo(ByteString.java:1331)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeBytesNoTag(CodedOutputStream.java:2751)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeBytes(CodedOutputStream.java:2724)
        at com.google.devtools.build.lib.worker.WorkerProtocol$Input.writeTo(WorkerProtocol.java:223)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeMessageNoTag(CodedOutputStream.java:2800)
        at com.google.protobuf.CodedOutputStream$OutputStreamEncoder.writeMessage(CodedOutputStream.java:2775)
        at com.google.devtools.build.lib.worker.WorkerProtocol$WorkRequest.writeTo(WorkerProtocol.java:1009)
        at com.google.protobuf.AbstractMessageLite.writeDelimitedTo(AbstractMessageLite.java:98)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.execInWorker(WorkerSpawnRunner.java:325)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.actuallyExec(WorkerSpawnRunner.java:167)
        at com.google.devtools.build.lib.worker.WorkerSpawnRunner.exec(WorkerSpawnRunner.java:121)
        at com.google.devtools.build.lib.exec.SpawnRunner.execAsync(SpawnRunner.java:218)
        at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:107)
        at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:76)
        at com.google.devtools.build.lib.exec.ProxySpawnActionContext.exec(ProxySpawnActionContext.java:44)
        at com.google.devtools.build.lib.analysis.actions.SpawnAction.internalExecute(SpawnAction.java:283)
        at com.google.devtools.build.lib.analysis.actions.SpawnAction.execute(SpawnAction.java:290)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.executeAction(SkyframeActionExecutor.java:889)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:854)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.call(SkyframeActionExecutor.java:787)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:541)
        at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:697)
        at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:240)
        at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:437)
        at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:368)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
---8<---8<--- End of exception details ---8<---8<---

---8<---8<--- Start of log, file at C:/_bazel/qukqnr66/bazel-workers/worker-3-Javac.log ---8<---8<---
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Unrecognized VM option 'CompactStrings'
---8<---8<--- End of log ---8<---8<---
Target //tensorflow/lite/java/demo/app/src/main:TfLiteCameraDemo failed to build
INFO: Elapsed time: 2.330s, Critical Path: 0.20s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

My envvars:

C:\src\tensorflow>set BAZEL
BAZEL_SH=C:\msys64\usr\bin\bash.exe
BAZEL_VC=C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC

C:\src\tensorflow>set JAVA
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_162

laszlocsomor on 28 Feb 2019

I finally figured out why.

Looks like for TensorFlow Lite we should not run the ./configure script. The generated .tf_configure.bazelrc file contains flags that would cause the build to fail.

Can you re-try after deleting .tf_configure.bazelrc file?

Update:
Problematic flags are:

--copt=-DWIN32_LEAN_AND_MEAN --host_copt=-DWIN32_LEAN_AND_MEAN
--distinct_host_configuration=false

meteorcloudy on 4 Mar 2019

❤1

Yes, that works around the crash, thank you!

The build now succeeds, though some actions print what look like errors:

INFO: From Validating Android resources for //tensorflow/lite/java:tensorflowlite:
Mar 04, 2019 2:11:26 PM com.google.devtools.build.android.AndroidManifest parseFrom
WARNING:
CONFIGURATION: bazel-out\android-armeabi-v7a-opt\bin\tensorflow\lite\java\tensorflowlite_processed_manifest\AndroidManifest.xml has no minSdkVersion. Using 1.
INFO: From Processing Android resources for //tensorflow/lite/java/demo/app/src/main:TfLiteCameraDemo:
Mar 04, 2019 2:12:10 PM com.google.devtools.build.android.AndroidDataMerger doMerge
WARNING:
CONFLICT: asset:WORKSPACE is provided with ambiguous priority from:
        external\tflite_mobilenet_quant\WORKSPACE
        external\tflite_mobilenet_float\WORKSPACE
Target //tensorflow/lite/java/demo/app/src/main:TfLiteCameraDemo up-to-date:
  C:/_bazel/qukqnr66/execroot/org_tensorflow/bazel-out/x64_windows-opt/bin/tensorflow/lite/java/demo/app/src/main/TfLiteCameraDemo_deploy.jar
  C:/_bazel/qukqnr66/execroot/org_tensorflow/bazel-out/x64_windows-opt/bin/tensorflow/lite/java/demo/app/src/main/TfLiteCameraDemo_unsigned.apk
  C:/_bazel/qukqnr66/execroot/org_tensorflow/bazel-out/x64_windows-opt/bin/tensorflow/lite/java/demo/app/src/main/TfLiteCameraDemo.apk
INFO: Elapsed time: 152.055s, Critical Path: 43.70s
INFO: 310 processes: 284 local, 26 worker.
INFO: Build completed successfully, 358 total actions

C:\src\tensorflow>echo %errorlevel%
0

laszlocsomor on 4 Mar 2019

CONFLICT: asset:WORKSPACE is provided with ambiguous priority from:
external\tflite_mobilenet_quant\WORKSPACE
external\tflite_mobilenet_float\WORKSPACE

@dslomov @dkelmer might know what does it mean?

meteorcloudy on 4 Mar 2019

That is an android error message. Adding @ahumesky, @jin in case what I'm about to say is wrong or if they have something more to add :)

The error message gets thrown here: https://github.com/bazelbuild/bazel/blob/a1f2ddb40ba57abd001449f0af80ca0357cabfbc/src/tools/android/java/com/google/devtools/build/android/UnwrittenMergedAndroidData.java#L151

(also see: https://github.com/bazelbuild/bazel/blob/a1f2ddb40ba57abd001449f0af80ca0357cabfbc/src/tools/android/java/com/google/devtools/build/android/MergeConflict.java#L31)

IIRC, assets are one form of data that your android app might use. You can provide multiple asset roots (directories), and then bazel will attempt to merge the assets from the roots for you. If there are conflicts, bazel will just pick one of them and keep going, but you will get warnings like the one you're seeing.

The WORKSPACE file is not an asset that is useful to an android app. So my guess is that tensorflow has (at least) two asset roots which contain useful assets, but which also contain a WORKSPACE file. This is triggering the warning. But the warning is almost certainly safe to ignore because I doubt the app is actually using either of the WORKSPACE files. However, if this is true, I do not have a suggestion for how to disable the warning.

Question: did you make a change that recently started triggering these warnings?

dkelmer on 4 Mar 2019

@dkelmer is correct. It's very likely that the WORKSPACE files are accidentally over-globbed into the assets dir, so this should be safe to ignore. We really want to turn this into an error, though, but we can't do that yet without a plan that would not break everything :-)