Serving: Executing genrule @com_github_libevent_libevent

[mshahriarinia]

Creating a Tensorflow serving in docker according to Serving Inception Model with TensorFlow Serving and Kubernetes:

• I use Dockerfile
• Get and run the docker container:

$ docker build --pull -t $USER/tensorflow-serving-devel -f tensorflow_serving/tools/docker/Dockerfile.devel .
$ docker run --name=inception_container -it $USER/tensorflow-serving-devel

• Get Tensorflow serving

$ git clone --recurse-submodules https://github.com/tensorflow/serving

• Get Tensroflow

$ cd serving && git clone --recursive https://github.com/tensorflow/tensorflow.git
$ cd tensorflow
$  ./configure

This complains that

You have bazel 0.5.4 installed.
Please upgrade your bazel installation to version 0.10.0 or higher to build TensorFlow!

So in the Dockerfile I modify ENV BAZEL_VERSION 0.5.4 to ENV BAZEL_VERSION 0.10.0 and redo the process, and configure according to all default values. Then

bazel test tensorflow_serving/...

and I get the error

ERROR: /root/.cache/bazel/_bazel_root/01a289b7faaf5ec651fb0e4e35f862a1/external/com_github_libevent_libevent/BUILD.bazel:52:1: Executing genrule @com_github_libevent_libevent//:libevent-srcs failed (Exit 127)
./autogen.sh: 18: ./autogen.sh: aclocal: not found
INFO: Elapsed time: 4282.912s, Critical Path: 153.66s
FAILED: Build did NOT complete successfully

I haven't found anything regarding com_github_libevent_libevent Any suggestions?

[eurus-yu]

I have same problem!

[netfs]

The Dockerfile does not have 'automake' and 'libtool' deb packages listed.

I will update the master branch with these shortly, until then you can add these two packages locally in the Dockerfile config and rebuild the image.

[mshahriarinia]

@netfs That was indeed the case as adding those two dependencies made the process keep going further. But I observe another strange error later on:

The process finishes by "Executed 2 out of 71 tests: 69 tests pass and 2 fail locally."

FAIL: //tensorflow_serving/util/net_http/server/internal:evhttp_request_test (see /root/.cache/bazel/_bazel_root/01a289b7faaf5ec651fb0e4e35f862a1/execroot/tf_serving/bazel-out/k8-fastbuild/testlogs/tensorflow_serving/util/net_http/server/internal/evhttp_request_test/test.log)
FAIL: //tensorflow_serving/util/net_http/server/internal:evhttp_server_test (see /root/.cache/bazel/_bazel_root/01a289b7faaf5ec651fb0e4e35f862a1/execroot/tf_serving/bazel-out/k8-fastbuild/testlogs/tensorflow_serving/util/net_http/server/internal/evhttp_server_test/test.log)

The two logs show:

root@5647334d48f9:~/serving# cat /root/.cache/bazel/_bazel_root/01a289b7faaf5ec651fb0e4e35f862a1/execroot/tf_serving/bazel-out/k8-fastbuild/testlogs/tensorflow_serving/util/net_http/server/internal/evhttp_request_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
-----------------------------------------------------------------------------
Running main() from test_main.cc
[==========] Running 5 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 5 tests from EvHTTPRequestTest
[ RUN      ] EvHTTPRequestTest.SimpleGETNotFound
[warn] getaddrinfo: address family for nodename not supported
[evhttp_server.cc : 217] RAW: Couldn't bind to port 0
Aborted

and

root@5647334d48f9:~/serving# cat /root/.cache/bazel/_bazel_root/01a289b7faaf5ec651fb0e4e35f862a1/execroot/tf_serving/bazel-out/k8-fastbuild/testlogs/tensorflow_serving/util/net_http/server/internal/evhttp_server_test/test.log
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
-----------------------------------------------------------------------------
Running main() from test_main.cc
[==========] Running 6 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 6 tests from EvHTTPServerTest
[ RUN      ] EvHTTPServerTest.AcceptingTerminating
[warn] getaddrinfo: address family for nodename not supported
[evhttp_server.cc : 217] RAW: Couldn't bind to port 0
Aborted

I made sure the container can ping outside websites and get replies

apt-get update
apt-get install iputils-ping
ping www.google.com

So what could be causing Couldn't bind to port 0?

[netfs]

thanks for reporting the test failures. this is due to our attempt to listen on ipv6:

https://github.com/tensorflow/serving/blob/c42810c1e149269d3bdeefa0fca56092215b1bb6/tensorflow_serving/util/net_http/server/internal/evhttp_server.cc#L214

in docker container that does not have ipv6 enabled. we should update
the code to not do that and work equally well in ipv4-only environment.

while we get the docker image/build issues resolved in this issue, can
you please file a separate issue for test failures (for better tracking).
will submit fixes against that issue, and have the tests pass.

[netfs]

fixed docker+bazel issues in master by:
https://github.com/tensorflow/serving/commit/daea4ec4892af0b54f29a81be7ebadf835af0ad5

feel free to reopen if the issue still persists

[Debasish-Das-CK]

Looks like my mac compilation is failing on this as well...Is it not suppose to work in mac ?

[lkluo]

same error with bazel 0.11.1

[Debasish-Das-CK]

are u building on mac OR through docker ?

[lkluo]

@Debasish-Das-CK: Neither, I am using CentOS.

[jiangcore]

@netfs I met the same problem in Bazel 0.14 under Ubuntu.

ERROR: /home/shaozhi/.cache/bazel/_bazel_shaozhi/0dd579deea072292803a0dc9d494ddde/external/com_github_libevent_libevent/BUILD.bazel:52:1: Executing genrule @com_github_libevent_libevent//:libevent-srcs failed (Exit 127)
./autogen.sh: 18: ./autogen.sh: aclocal: not found

[netfs]

since new release 1.8.0 (and this applies to master too), you need automake and libtool packages to build. see the complete dependencies here:

https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/setup.md

OR here:

https://github.com/tensorflow/serving/blob/0346f182fea15bb643bd699d228242d8072a6fd2/tensorflow_serving/tools/docker/Dockerfile.devel#L18

(and if feasible, build using docker, so you dont have to worry about deps and bazel version).

[zizihaier]

@netfs I met the similar problem in Bazel 0.24.0 under Ubuntu and tensorflow serving r1.14 when execute '/bin/sh -c bazel build --color=yes --curses=yes ${TF_SERVING_BAZEL_OPTIONS} --verbose_failures --output_filter=DONT_MATCH_ANYTHING ${TF_SERVING_BUILD_OPTIONS} tensorflow_serving/model_servers:tensorflow_model_server && cp bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server /usr/local/bin/ '

ERROR:
cp: cannot stat '/root/.cache/bazel/_bazel_root/e53bbb0b0da4e26d24b415310219b953/sandbox/processwrapper-sandbox/969/execroot/tf_serving/external/com_github_libevent_libevent/*': No such file or directory
Target //tensorflow_serving/model_servers:tensorflow_model_server failed to build

[byronyi]

@netfs I met the similar problem in Bazel 0.24.0 under Ubuntu and tensorflow serving r1.14 when execute '/bin/sh -c bazel build --color=yes --curses=yes ${TF_SERVING_BAZEL_OPTIONS} --verbose_failures --output_filter=DONT_MATCH_ANYTHING ${TF_SERVING_BUILD_OPTIONS} tensorflow_serving/model_servers:tensorflow_model_server && cp bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server /usr/local/bin/ '

ERROR:
cp: cannot stat '/root/.cache/bazel/_bazel_root/e53bbb0b0da4e26d24b415310219b953/sandbox/processwrapper-sandbox/969/execroot/tf_serving/external/com_github_libevent_libevent/*': No such file or directory
Target //tensorflow_serving/model_servers:tensorflow_model_server failed to build

Same here using sandboxed remote build execution. Gently ping @gautamvasudevan.