Bazel: Bazel HEAD breaks Tensorflow and tensorflow/serving.

[buchgr]

This error only happens with Bazel from head (2a2aa7ab31367da47f8449059fdfcddfba3a20f1). Works with 10.1.

ERROR: /var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/e276c88c2550e615b3487b1506ef1c36/external/jpeg/BUILD:126:12: Illegal ambiguous match on configurable attribute "deps" in @jpeg//:jpeg:
--
  | [2018-02-18T17:53:11Z] @jpeg//:armeabi-v7a
  | [2018-02-18T17:53:11Z] @jpeg//:k8
  | [2018-02-18T17:53:11Z] Multiple matches are not allowed unless one is unambiguously more specialized.

See https://buildkite.com/bazel/bazel-downstream/builds/84#b82d6d34-8932-40cf-8b71-5ca793480f2f

@vladmos @laurentlb @dslomov any thoughts?

[gregestren]

I don't know specifically what's going on, but since @jpeg//:k8 and @jpeg//:armeabi-v7a trigger on --cpu and --android_cpu respectively, it's perfectly easy to trigger both at the same time, producing the above error:

$ bazel build //foo --cpu=k8 --android_cpu=armeabi-v7a

As for why this only affects Bazel at head, this could easily be related to changes in Android rule configuration transition logic. Will dig in more...

[katre]

Marking as a release blocker while @gregestren checks for the root cause. Might need to cherry-pick this into #3959.

[gregestren]

This is due to https://github.com/bazelbuild/bazel/commit/fc30733995afd1a104051a73218b7a40a0428eb9, which changes the default value of --android_cpu to armeabi-v7a.

@katre found a minimal repro:

$ bazel_at_head build --nobuild @jpeg//:jpeg

@buchgr: how important is it that @jpeg://armeabi-v7a triggers on android_cpu? The relationship between android_cpu and cpu is subtle and confusing, but most of the time when you want to trigger an architecture-dependent library you're better off using cpu.

Is there someone on Tensorflow who can consider this?

[buchgr]

@gregestren I have no idea. All I know is that it breaks TensorFlow's build. Is it possible to fix this in the TensorFlow repository so that it works with the current Bazel version and Bazel HEAD?

[gregestren]

+@jart on Tensorflow.

See also the bug that created that rule: https://github.com/tensorflow/tensorflow/issues/4807 and the fix: https://github.com/tensorflow/tensorflow/commit/52ffc792fdef41de20ac60dc61d4e809d83b558d.

[katre]

I don't believe this issue should be a release blocker for 0.11.0 since it looks like the fix is on Tensorflow's side. Any problems with that?

[buchgr]

@gregestren @katre sounds good. can you open a bug against tensorflow describing the fix please?

[katre]

Tensorflow appears to have removed jpeg.BUILD (I can't find it at HEAD, anyway), and the build is green, so closing this issue.

[buchgr]

The error is still happening: https://buildkite.com/bazel/bazel-with-downstream-projects-bazel/builds/141#82dedeac-aa58-46f9-99bf-d03d8da5a03e

[xingao267]

We are seeing this error in our tensorflow remote tests using Bazel 0.12.0rc1 or HEAD. It is ok with Bazel 0.11.1.

https://sponge.corp.google.com/target?id=b099f31b-f2ac-40a5-ba50-88503ffae0f7&target=alphasource-toolchains%2Fnightly%2Fremotebuild-test%2Ftensorflow-host_xenial-exec_debian8-clang-bazel_0120&searchFor=

and here https://kokoro.corp.google.com/job/alphasource-toolchains/job/nightly/job/remotebuild-test/ (search tensorflow)

[meteorcloudy]

@xingao267 Are you building TensorFlow@HEAD, https://github.com/tensorflow/tensorflow/pull/17508 should have fixed the problem.

[xingao267]

@meteorcloudy Thanks for the pointer. Indeed, we are building TF at a commit before the fix.

[lfpino]

This is still happening according to https://buildkite.com/bazel/bazel-with-downstream-projects-bazel/builds/177#10f9cdaa-75bc-416a-b635-633153fa1e8b, @meteorcloudy can you please follow up?

[meteorcloudy]

Sure, filed an issue to TensorFlow_serving, following up at https://github.com/tensorflow/serving/issues/825

[Bardo91]

O had the same problem while compiling TF as a shared library:
bazel build //tensorflow:libtensorflow.so --cpu=k8

WARNING: /home/bardo91/.cache/bazel/_bazel_bardo91/650966360851c5a6b9fcb8e658f1efec/external/protobuf_archive/WORKSPACE:1: Workspace name in /home/bardo91/.cache/bazel/_bazel_bardo91/650966360851c5a6b9fcb8e658f1efec/external/protobuf_archive/WORKSPACE (@com_google_protobuf) does not match the name given in the repository's definition (@protobuf_archive); this will cause a build error in future versions
ERROR: /home/bardo91/.cache/bazel/_bazel_bardo91/650966360851c5a6b9fcb8e658f1efec/external/jpeg/BUILD:415:11: Illegal ambiguous match on configurable attribute "cmd" in @jpeg//:configure:
@jpeg//:k8
@jpeg//:armeabi-v7a
Multiple matches are not allowed unless one is unambiguously more specialized.
ERROR: Analysis of target '//tensorflow:libtensorflow.so' failed; build aborted: 

/home/bardo91/.cache/bazel/_bazel_bardo91/650966360851c5a6b9fcb8e658f1efec/external/jpeg/BUILD:415:11: Illegal ambiguous match on configurable attribute "cmd" in @jpeg//:configure:
@jpeg//:k8
@jpeg//:armeabi-v7a
Multiple matches are not allowed unless one is unambiguously more specialized.
INFO: Elapsed time: 0.926s
FAILED: Build did NOT complete successfully (25 packages loaded)

I removed all the references of "armeabi-v7a" in the file thirdparty/jpeg/jpeg.BUILD and now it seems to work, but I am not quite sure how bazel works. Is it a bug?