Bazel: tensorflow building issue on ubuntu 14.04: Accessing remote repository through a proxy

[lacartelacarte]

As (https://github.com/tensorflow/tensorflow/issues/67) is declared as Bazel issue I'm writing here. So I'm having the same issue with the original post. As mentioned in the original post, by manually wget and placing the jpeg library the error message about jpeg has gone. However there are still many error messages, and by trying multiple times the error message seems to change but always fails with downloading something. These are the variations.

Variation 1:
ERROR: /home/name/work/tensorflow/tensorflow/core/platform/default/build_config/BUILD:36:1: no such package '@re2//': https://github.com/google/re2.git: cannot open git-upload-pack and referenced by '//tensorflow/core/platform/default/build_config:platformlib'.

Variation 2:
ERROR: /home/name/work/tensorflow/tensorflow/core/platform/default/build_config/BUILD:36:1: no such package '@png_archive//': Error downloading from https://storage.googleapis.com/libpng-public-archive/libpng-1.2.53.tar.gz to /home/name/.cache/bazel/_bazel_name/894ee6ce07893c5c665fe600d592867f/external/png_archive: Error downloading https://storage.googleapis.com/libpng-public-archive/libpng-1.2.53.tar.gz to /home/name/.cache/bazel/_bazel_name/894ee6ce07893c5c665fe600d592867f/external/png_archive/libpng-1.2.53.tar.gz: storage.googleapis.com and referenced by '//tensorflow/core/platform/default/build_config:platformlib'.

Variation 3:
ERROR: /home/name/work/tensorflow/tensorflow/tensorflow.bzl:224:3: no such package '@gemmlowp//': https://github.com/google/gemmlowp.git: cannot open git-upload-pack and referenced by '//tensorflow/core:kernels'.

[damienmg]

Wow, do you have special network configuration? It seems like it cannot fetch remote dependencies.

@kchodorow @davidzchen might have more idea on the problem source.

[lacartelacarte]

@damienmg Thank you for reply. Yes it seems to be network problem. My working machine is behind a proxy but I think I have correct setting to pass it. At least I could manually wget the jpeg library without problem. I'll check with each of the libraries mentioned in the error message when I have access to the machine tomorrow, but I think there won't be much problem by manual wget. Does Bazel need special setting for proxy?

[damienmg]

Oops assigning to Kristina then

[jgbos]

FYI, I'm having the same issue behind a proxy set as http://site.com:port (same for https).

• I'm using Centos 6.7
• Proxy settings for both HTTP_PROXY and HTTPS_PROXY are set (and lowercase versions)
• If I download the files for new_http_archive the same issue appears for git_repository
• Global config for proxy is set for git

[kchodorow]

Unfortunately, there is no support at the moment for fetching external resources through a proxy. It looks like we'd want to add a mechanism to specify local settings, which I don't think Bazel has at the moment.

[lacartelacarte]

Just confirmed I can git clone and wget those resources without problem, but fails with Bazel. Are there any work around to let Bazel use the system's proxy setting?

[kchodorow]

No (although we should add that!), but you should be able to change your WORKSPACE file to local_repository() or new_local_repository() once they're downloaded to your local machine (see http://bazel.io/docs/be/workspace.html#local_repository, http://bazel.io/docs/be/workspace.html#new_local_repository).

[lacartelacarte]

Got it. Thank you for the information. So the current best practice would be:

1. try building
2. read error message for missing library
3. download it and keep it somewhere (as local repository)
4. goto 1 until no error appears
   I'll try this later maybe.

[ulfjack]

Does java.net.URL use the proxy settings?

[philwo]

Java seems to support HTTP(S) proxies transparently: https://docs.oracle.com/javase/7/docs/api/java/net/doc-files/net-properties.html

We should make sure that our fetching code works with them (I haven't tried whether it does) and optionally on server start-up check whether these variables are empty, but the usual environment variables are set (HTTP_PROXY, HTTPS_PROXY) and then set the Java properties correctly from that.

[srsaharoy]

Hi,
After using local_repository() for libpng, jpegsrc, re2 and gemmlowp packages, the earlier proxy issue went away, but the build is now failing due to 'cycle in dependency graph', as shown below:

$ bazel build -c opt //tensorflow/cc:tutorials_example_trainer
Extracting Bazel installation...
.......
ERROR: /home/ssaharo/.cache/bazel/_bazel_ssaharo/64bf5fcc928610d1fc32c86de697b420/external/png_archive/BUILD:23:1: in genrule rule @png_archive//:configure: cycle in dependency graph:
//tensorflow/cc:tutorials_example_trainer
//tensorflow/core:tensorflow
//tensorflow/core:tensorflow_opensource
//tensorflow/core:core
//tensorflow/core:core_cpu
//tensorflow/core:framework
//tensorflow/core:lib
//tensorflow/core/platform/default/build_config:platformlib
@png_archive//:png

• @png_archive//:configure [self-edge].
  ERROR: Loading of target '//tensorflow/cc:tutorials_example_trainer' failed; build aborted.
  ERROR: Loading failed; build aborted.

[zhanmusi1207]

met the same problem as @srsaharoy ...

[srsaharoy]

If I change the 'genrule' parameters in png.BUILD and jpeg.BUILD to specifically include "_._" as follows

from:

genrule(
name = "configure",
srcs = glob(

["*_/_"],

to:

genrule(
name = "configure",
srcs = glob(

["_/_."],

then the build no longer fails with error "cycle in dependency graph"

However, now it fails because it can't find a source file (which does exist!) in either libpng-1.2.53 or jpeg-9a directory path. New error message is below which says that it could not find "libpng-1.2.53/pngpread.c". I also do not understand how or who is
generating signal 15 (sigterm) - is it generated because one thread of build process failed due to missing file? I did execute "bazel clean --expunge" before starting the build...

$ bazel build -c opt //tensorflow/cc:tutorials_example_trainer
......
INFO: Found 1 target...
ERROR: missing input file '@png_archive//:libpng-1.2.53/pngpread.c'.
ERROR: /home/ssaharo/tensorflow/tensorflow/cc/BUILD:28:1: Creating runfiles tree bazel-out/host/bin/tensorflow/cc/ops/array_ops_gen_cc.runfiles [for host] failed: build-runfiles failed: error executing command /home/ssaharo/.cache/bazel/_bazel_ssaharo/64bf5fcc928610d1fc32c86de697b420/tensorflow/_bin/build-runfiles bazel-out/host/bin/tensorflow/cc/ops/array_ops_gen_cc.runfiles_manifest ... (remaining 1 argument(s) skipped): com.google.devtools.build.lib.shell.AbnormalTerminationException: Process terminated by signal 15; also encountered an error while attempting to retrieve output.
ERROR: /home/ssaharo/.cache/bazel/_bazel_ssaharo/64bf5fcc928610d1fc32c86de697b420/external/png_archive/BUILD:33:1: @png_archive//:png: missing input file '@png_archive//:libpng-1.2.53/pngpread.c'.
Target //tensorflow/cc:tutorials_example_trainer failed to build
Use --verbose_failures to see the command lines of failed build steps.
ERROR: /home/ssaharo/.cache/bazel/_bazel_ssaharo/64bf5fcc928610d1fc32c86de697b420/external/png_archive/BUILD:33:1 1 input file(s) do not exist.

[srsaharoy]

sorry, somehow made a typo while posting:
the change was from "/*" to "/_._"

[lacartelacarte]

I tried the 'read error -> download lib -> set local repository' loop.
After fixing these reference issues I could build tensorflow with bazel without problem.
My protobuf's version was old and this became a problem when I tried to import tensorflow in python, but by replacing protobuf with >= 3.0 this problem is fixed and so finally I can test some network with tensorflow.

[uriv]

could anyone who succeeded to overcome this problem post their changes to the WORKSPACE file?

[damienmg]

See issue #623 srsaharoy posted a WORKSPACE file (which only trouble is the circular dependencies and the fix is in the bug report).

We should probably provide a WORKSPACE + shell script to work around that bug.

[uriv]

Thanks. I added a summary of the fix to that issue (https://github.com/bazelbuild/bazel/issues/623#issuecomment-158151936).

[kbrems]

Note, the posted WORKSPACE file only gets you past building the tensorflow tutorials_example_trainer. If you want to rebuild the whole package and install, for example to create a new TensorFlow user operator, there are no less than 74 external dependencies to manually download and edit in your WORKSPACE file...(mostly needed for TensorBoard). This is not very workable for those of us trying to build TensorFlow from behind a corporate firewall.

[flawless2011]

Has the code fix in 5fd83b2 been tested? It doesn't seem to work for me after I build bazel from source and use that to do my tensorflow build. I've set the $HTTP_PROXY and $HTTPS_PROXY env vars with my proxy info http://me:[email protected]:8000 (scrubbed to protect the innocent). I get an error message like this:

/Users/me/Dev/tensorflow/tensorflow/third_party/eigen3/BUILD:3:1: no such package '@eigen_archive//': Error downloading from https://bitbucket.org/eigen/eigen/get/3.3-beta1.tar.gz to /private/var/tmp/_bazel_me/7c952997242db15a11242e48dacb764f/external/eigen_archive: Error downloading https://bitbucket.org/eigen/eigen/get/3.3-beta1.tar.gz to /private/var/tmp/_bazel_me/7c952997242db15a11242e48dacb764f/external/eigen_archive/3.3-beta1.tar.gz: http://me:[email protected] and referenced by '//third_party/eigen3:eigen3'.

Not sure why the proxy hostname is being printed out in the error (shouldn't do this since it contains credentials). Also the proxy port is stripped from that error message so not sure if that's an issue or not.

I've glanced through the Java code on commit 5fd83b2 and don't see any immediate issues but I admittedly don't know much about the rest of the code around the HttpDownloader class.

Using bazel 0.1.2 from the installer (which does not have proxy support) results in this error message:
/Users/me/Dev/tensorflow/tensorflow/third_party/eigen3/BUILD:3:1: no such package '@eigen_archive//': Error downloading from https://bitbucket.org/eigen/eigen/get/3.3-beta1.tar.gz to /private/var/tmp/_bazel_me/7c952997242db15a11242e48dacb764f/external/eigen_archive: Error downloading https://bitbucket.org/eigen/eigen/get/3.3-beta1.tar.gz to /private/var/tmp/_bazel_me/7c952997242db15a11242e48dacb764f/external/eigen_archive/3.3-beta1.tar.gz: Connection refused and referenced by '//third_party/eigen3:eigen3'.

Any thoughts?

[uriv]

@flawless2011 There is info on how to overcome the proxy error by manually downloading the missing libs at https://github.com/bazelbuild/bazel/issues/623#issuecomment-158984782

[flawless2011]

@uriv I understand but I'm trying to find out if the code fix committed by @kchodorow is complete or not. The workaround in #623 is only a temporary solution vs. actually fixing the problem with the proxy.

[nitingupta910]

I just cloned and built bazel from source but still can't build tensorflow due to bazel failing to download dependencies behind proxy. Set all variables: http_proxy, https_proxy, HTTP_PROXY, HTTPS_PROXY. Downloads directly using wget work. Manual workflow as described in #623 is way too cumbersome, so please please fix bazel proxy issue.

[damienmg]

Reopening so @kchodorow can investigate.

[kchodorow]

Not sure why the proxy hostname is being printed out in the error (shouldn't do this since it contains credentials). Also the proxy port is stripped from that error message so not sure if that's an issue or not.

The proxy solution was supposed to be complete, but it's possible I missed something. On the plus side, obviously it's doing something with the proxy and e1de068273cd8e72f34410526f27c779a8b69f30 should at least give a better error message when the download fails. Let me try to reproduce and I'll get back to you.

[kchodorow]

Oh, looks like I need to add support for username/password authentication.

[flawless2011]

@kchodorow I'm still not able to get either new_http_archive or git_repository external dependencies to work with the latest from master. I've tried setting HTTP_PROXY and HTTPS_PROXY (and http_proxy and https_proxy for giggles) with our corporate proxy (requires id/password) and also using a local workstation proxy (doesn't require id/password). Both fail with same error message.

Corporate proxy (http://me:[email protected]:8000):
ERROR: /Users/me/Dev/tensorflow/tensorflow/third_party/eigen3/BUILD:3:1: no such package '@eigen_archive//': Error downloading from https://bitbucket.org/eigen/eigen/get/5651786.tar.gz to /private/var/tmp/_bazel_me/7c952997242db15a11242e48dacb764f/external/eigen_archive: Error downloading https://bitbucket.org/eigen/eigen/get/5651786.tar.gz to /private/var/tmp/_bazel_me/7c952997242db15a11242e48dacb764f/external/eigen_archive/5651786.tar.gz: http://me:[email protected] and referenced by '//third_party/eigen3:eigen3'.

Local proxy (http://localhost:3232):
ERROR: /Users/me/Dev/tensorflow/tensorflow/third_party/eigen3/BUILD:3:1: no such package '@eigen_archive//': Error downloading from https://bitbucket.org/eigen/eigen/get/5651786.tar.gz to /private/var/tmp/_bazel_me/7c952997242db15a11242e48dacb764f/external/eigen_archive: Error downloading https://bitbucket.org/eigen/eigen/get/5651786.tar.gz to /private/var/tmp/_bazel_me/7c952997242db15a11242e48dacb764f/external/eigen_archive/5651786.tar.gz: http://localhost and referenced by '//third_party/eigen3:eigen3'.

I've been trying to get Eclipse setup to debug your code to see if I can sort something out there but I can't even get the scripts/setup-eclipse.sh setup script to run to completion. Fails out with similar dependency retrieval error trying to get some golang dependency.
ERROR: /Users/me/Dev/opensource/bazel/tools/build_rules/go/toolchain/BUILD:15:1: no such package '@golang-darwin-amd64//': Error downloading from https://storage.googleapis.com/golang/go1.5.1.darwin-amd64.tar.gz to /private/var/tmp/_bazel_me/bca7eb51f6c9d1c3daf3d31744a7e207/external/golang-darwin-amd64: Error downloading https://storage.googleapis.com/golang/go1.5.1.darwin-amd64.tar.gz to /private/var/tmp/_bazel_me/bca7eb51f6c9d1c3daf3d31744a7e207/external/golang-darwin-amd64/go1.5.1.darwin-amd64.tar.gz: http://localhost and referenced by '//tools/build_rules/go/toolchain:toolchain'.

@kchodorow I know Google doesn't use a proxy server but can you set one up on your workstation locally and try building something like tensorflow that has external dependencies to see if you can reproduce my problem?

[kchodorow]

It looks like you're not running the latest version of bazel (I changed the error message for download failures in e1de068273cd8e72f34410526f27c779a8b69f30). Could you try pulling HEAD, building //src:bazel, and running your build using bazel-bin/src/bazel?

[flawless2011]

I am on HEAD for master. I've been building source using ./compile.sh and then running my build with ./output/bazel executable. I'll try using bazel build instead and running with ./bazel-bin/src/bazel.

[flawless2011]

Same error. Looking through the commit log I don't see your e1de068 on master.

[flawless2011]

OK after looking a little more closely I do see that commit in the log (further back than I expected). So yes I have that code in my current build but I'm not seeing any change in the error message.

[jrings]

If I see correctly, if the proxy URL doesn't start with https, it's rejected as invalid. This is a problem for proxies that are http only, even for https URLs.

EDIT: Actually I'm wrong, it has https? in the regexp. Nevertheless, the proxy address is refused

ERROR: /home/ec2-user/install/tensorflow/tensorflow/core/platform/default/build_config/BUILD:36:1: 
no such package '@jpeg_archive//': Error downloading from http://www.ijg.org/files/jpegsrc.v9a.tar.gz 
to /home/ec2-user/.cache/bazel/_bazel_ec2-
user/51f5e1bb7fe78e63c0affed5945b933c/external/jpeg_archive: Error 
downloading http://www.ijg.org/files/jpegsrc.v9a.tar.gz to /home/ec2-user/.cache/bazel/_bazel_ec2-
user/51f5e1bb7fe78e63c0affed5945b933c/external/jpeg_archive/jpegsrc.v9a.tar.gz: Proxy address
 http://USER:[email protected]:1234/ is not a valid URL and referenced by
 '//tensorflow/core/platform/default/build_config:platformlib'.

EDIT 2: Got a different error this time:

Error parsing proxy port: http://USER:[email protected]:1234/ and referenced by '//tensorflow/core/platform/default/build_config:platformlib'.

So apparently the trailing / disturbed the port detection. I changed the URL and now I get the same error @flawless2011 posted :/

[flawless2011]

I have a fix for the proxy problem in HttpDownloader but that only seems to fix downloading the new_http_archive dependencies and not the re2 git dependency. I guess I'll work on getting through the contribution workflow tomorrow so you guys can review the code change. Then I'll try to sort out why the re2 dependency is still failing.

[nsuke]

@flawless2011 git dependencies use JGit so HttpDownloader is not related.

bazel --host_jvm_args=-Dhttp.proxyHost=my.proxy -Dhttp.proxyPort=8888 -Dhttps.proxyHost=....

fixed the problem for me. I had no luck with new_http_archive dependencies.

[kchodorow]

@flawless2011: I have totally failed to get a proxy set up so far, do you have a fork somewhere where I can take a look at how you got it working? It is not surprising that re2 isn't working, git_repository and maven_jar use different downloading code than the http rules, so those will have to be fixed separately (but it shouldn't be too hard once I know what's wrong with the current code).

[flawless2011]

@kchodorow The code is all uncommitted on my workstation. The Google CLA has thrown things off a bit because I'm working with our internal legal processes to assess. We're not used to dealing with a CLA on open source contributions.

Easiest thing might be for me to fork, commit, and send you guys a PR so you can see the code change while the CLA is being looked at. I know that's not exactly following your contribution guidance but hopefully it will do for now.

I've made changes to HttpDownloader, GitCloner, and moved the create proxy stuff out to a new ProxyHelper class. I think if you guys are agreeable to the solution, I will do a bit more cleanup and refactoring.

[flawless2011]

@kchodorow Here's our fork with the changes to enable proper proxy access: https://github.com/StateFarmIns/bazel

[jmtatsch]

@kchodorow I can confirm @flawless2011 fix works for me

[ansukla]

There are still a few problems with the HTTPDownloader code:

1. Ignores no_proxy
2. Creates a java proxy object with server name http://server and port. I had to change the code to use protocol.length() + 3 as offset instead of 0
3. Expects https proxy server name to start with https:// - not always true

[flawless2011]

@ansukla Are you using my fork that contains the proxy fixes? https://github.com/StateFarmIns/bazel

[ansukla]

@flawless2011 No I thought your code was already merged. I just quickly browsed through ProxyHelper.java and it looks good to me except 1 and 3 are probably still not addressed. Hope your code will make it through - there is no way for the original code to ever work with proxy and it fails silently.

[flawless2011]

Your concern about no_proxy I will look at further. I think 3 should work because my own proxy server uses an http proxy for both http and https resources. Let me know if you run it and see any issues.

[kchodorow]

I can work on no_proxy on our side, too. Let us know when CLA gets approved so we can get this merged!

[flawless2011]

Yeah sorry it's taking so long, our lawyers have a big backlog.

[kbrems]

@flawless2011 - what version of bazel is your fork based on? The latest tensorflow requires bazel 0.1.5 which definitely does not have a working proxy fix despite what is stated in the "important changes" section.

[kbrems]

I just confirmed that the StateFarm fork works both with our proxy and with the latest tensorflow. Thanks @flawless2011!

[hhclam]

I created a change to fix this issue: https://bazel-review.googlesource.com/#/c/2981/

[hhclam]

Sorry I didn't realize flawless2011@ had a more complete fix. Is that being merged?

The change I uploaded didn't work for jgit. I abandoned the review.

[kchodorow]

Yes, @flawless2011's fork is being merged, we were just waiting on the CLA.

[myme5261314]

@kchodorow
Have the proxy related issue been fixed in the HEAD?
I'm using the HEAD to build tensorflow with the command below:

http_proxy=http://localhost:port https_proxy=http://localhost:port bazel build -c opt --config=cuda //tensorflow/tools/pip_package:build_pip_package

But seems bazel doesn't use proxy. Since I could clone the boringssl repo locally through git with proxy.

tensorflow/tensorflow/workspace.bzl:84:3: no such package '@grpc//': Error cloning repository: https://boringssl.googlesource.com/boringssl: cannot open git-upload-pack caused by https://boringssl.googlesource.com/boringssl: cannot open git-upload-pack caused by https://boringssl.googlesource.com/boringssl: cannot open git-upload-pack and referenced by '//external:grpc_cpp_plugin'.
ERROR: Loading failed; build aborted.
INFO: Elapsed time: 3473.865s

[kchodorow]

Yes, the patch has been merged in. The proxy code looks for either HTTP_PROXY or HTTPS_PROXY (not the lowercase version). Although perhaps we should add support for the lowercase option.

[anshumang]

Running into a related issue it seems. Getting the error -
tensorflow/core/platform/default/build_config/BUILD:46:1: no such package '@re2//': Error cloning repository: https://github.com/google/re2.git: cannot open git-upload-pack caused by https://github.com/google/re2.git: cannot open git-upload-pack caused by https://github.com/google/re2.git: cannot open git-upload-pack and referenced by '//tensorflow/core/platform/default/build_config:platformlib'.

My command line is
sudo HTTP_PROXY=http://someproxy HTTPS_PROXY=https://someproxy /path-to-bazel-bin/bazel build -c opt --config=cuda //tensorflow/cc:tutorials_example_trainer

My bazel cache shows the following contents-
bazel_tools eigen_archive jpeg_archive local_config_cc local_jdk png_archive re2

Bazel Build label: 0.2.2b-jdk7

I can manually clone from https://github.com/google/re2.git from inside the same network.

Can someone pls point me to what I am missing here?

[chen2000]

getting the same error:
ERROR: no such package '@re2//': Error cloning repository: ssh://[email protected]/google/re2.git: Auth cancel caused by ssh://[email protected]/google/re2.git: Auth cancel caused by ssh://[email protected]/google/re2.git: Auth cancel and referenced by '//syntaxnet:base'.
ERROR: Loading failed; build aborted.
INFO: Elapsed time: 0.130s
ERROR: Couldn't start the build. Unable to run tests.

I cloned re2 to local. Is there anyway to build by the local clone?

[chen2000]

Also, I do not think it is a proxy problem.
I tried both my home network and workingplance network. Both of them met the error:

"ERROR: no such package '@re2//': Error cloning repository: ssh://[email protected]/google/re2.git: Auth cancel caused by ssh://[email protected]/google/re2.git: Auth cancel caused by ssh://[email protected]/google/re2.git: Auth cancel and referenced by '//syntaxnet:base'."

But I can successfully clone the repo from both the networks.

[kadrach]

Ending with a similar error on my end:

ERROR: /home/ubuntu/tensorflow/tensorflow/core/platform/default/build_config/BUILD:37:1: no such package '@re2//': Proxy address http://10.74.1.25:8080/ is not a valid URL and referenced by '//tensorflow/core/platform/default/build_config:platformlib'.

Proxy is set system wide (/etc/environment), tensorflow r0.8 and bazel HEAD(#c728a63).

_Edit:_ Turns out all that was needed was to remove the final / from the proxy definition, eg. http://1.2.3.4:8080/ => http://1.2.3.4:8080, as previously mentioned in this thread.

[ch3njust1n]

Has this been fixed yet or is there a workaround?

[damienmg]

This should be fixed at HEAD but not yet in the release, you can try the release candidate at https://storage.googleapis.com/bazel/0.3.0/rc2/index.html

[ch3njust1n]

When I run the command:
bazel test syntaxnet/... util/utf8/...
I get the following issues still:

models/syntaxnet/syntaxnet/BUILD:74:1: no such package '@re2//': Error cloning repository: https://github.com/google/re2.git: cannot open git-upload-pack caused by https://github.com/google/re2.git: cannot open git-upload-pack caused by https://github.com/google/re2.git: cannot open git-upload-pack and referenced by '//syntaxnet:base'.
ERROR: Loading failed; build aborted.
INFO: Elapsed time: 127.561s
ERROR: Couldn't start the build. Unable to run tests.

[dmitris]

I've been having the same or similar issue until I commented out in my .gitconfig:

[url "[email protected]:"]
       insteadOf = https://github.com/

[jmugan]

Yeah, I think my problem is related. I get

~/workspace/models/syntaxnet$ bazel test syntaxnet/... util/utf8/...
ERROR: /home/jmugan/workspace/models/syntaxnet/tensorflow/tensorflow/workspace.bzl:84:3: no such package '@gmock_archive//': Error downloading from https://archive.openswitch.net/gmock-1.7.0.zip to /home/jmugan/.cache/bazel/_bazel_jmugan/1fd2d6dd907f68126bdeed691eb1f38b/external/gmock_archive: Error downloading https://archive.openswitch.net/gmock-1.7.0.zip to /home/jmugan/.cache/bazel/_bazel_jmugan/1fd2d6dd907f68126bdeed691eb1f38b/external/gmock_archive/gmock-1.7.0.zip: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target and referenced by '//external:gtest'.
ERROR: Loading failed; build aborted.
INFO: Elapsed time: 3.211s
ERROR: Couldn't start the build. Unable to run tests.

[phact]

@jmugan I'm having the same problem, did you figure out a solution?

[bsilver8192]

@jmugan and @phact: I think that's a different error. I've seen similar messages on Debian due to https://bugs.launchpad.net/ubuntu/+source/ca-certificates-java/+bug/1396760, which requires running sudo /var/lib/dpkg/info/ca-certificates-java.postinst configure to work around.

If that doesn't fix your problem, one of you should open a new bug.

[jmugan]

@bsilver8192 Thanks for the suggestion, but that didn't fix it. I still get the exact same error.

[xuanchien]

I am also getting the same issue as @jmugan on my Mac OSX.

[sunil3590]

@jmugan @xuanchien I faced the same issue on ElCapitan. To solve it, I did the following (hack)

1. cd some_safe_folder
2. wget https://archive.openswitch.net/gmock-1.7.0.zip
3. python -m SimpleHTTPServer 8000
4. Change line 79 of models/syntaxnet/tensorflow/tensorflow/workspace.bzl from url = "https://archive.openswitch.net/gmock-1.7.0.zip" to url = "http://localhost:8000/gmock-1.7.0.zip"
   Basically what you're doing is, downloading the required package manually and setting up a server to serve the package from your local machine.

P.S - before you try what I suggested, you might want to try http instead of https in url = "https://archive.openswitch.net/gmock-1.7.0.zip". I haven't tried this though.

[jmugan]

@sunil3590 That did it! Thanks! I tried the switch to http, and that didn't work. I had to run the SimpleHTTPServer like you suggested. I got another error about undefined reference to symbol ceil@@GLIBC_2.2.5 but I'm sure that's another issue in another place. Thanks!

[xuanchien]

Thanks @sunil3590, that does the trick

[kimjunh]

Has there been a solution for this?
I'm getting the same error as @jmugan but I cannot use the proposed solution of downloading it manually and serving it locally since I'm running an isolated build from a docker

[kimjunh]

Also @sunil3590 can you specify what file you changed when you say change line 79?

[sunil3590]

@kimjunh line 79 of models/syntaxnet/tensorflow/tensorflow/workspace.bzl

[hughperkins]

Hi @sunil3590 I really like your idea, and created a repo, with a script to facilitate slightly https://github.com/hughperkins/tf_cached_build

[damienmg]

Do you know that you could simply does a bazel fetch --experimental_repository_cache /path/to/cache //... and then ship the
/path/to/cache directory and bazel build --experimental_repository_cache /path/to/cache //... would use that cache?

On Sun, May 28, 2017 at 8:17 AM Hugh Perkins [e-mail]
wrote:

Hi @sunil3590 https://github.com/sunil3590 I really like your idea, and
created a repo, with a script to facilitate slightly
https://github.com/hughperkins/tf_cached_build

—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/587#issuecomment-304521353,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADjHf8P5rSFL3dK5-dfjfZkXevDLyiYhks5r-ZAlgaJpZM4GgBZt
.

[hughperkins]

@damienmg I did not. Extremely useful information :-)

[damienmg]

:)

On Sun, May 28, 2017 at 8:36 AM Hugh Perkins [e-mail]
wrote:

@damienmg https://github.com/damienmg I did not. Extremely useful
information :-)

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/587#issuecomment-304522355,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADjHf3oSdH1P-J4O5djngZK5YwGA_L1tks5r-ZSHgaJpZM4GgBZt
.

[hughperkins]

By the way, @damienmg what version is required for this fuctionality to work?

[hughperkins]

Hi @damienmg I tried upgrading to bazel 0.5.0, but doesnt seem to work for me for tensorflow?

(basically what I did was:

• modify ./configure to use the --experimental_repository_cache command, in the fetch
• with wifi on, run ./configure, wait for it to finish
• weirdly, the cache was only 17MB at this point. I was expecting it to be much larger?
• then I started running a build for a few minutes
• stopped the build, turned off wifi, redid ./configure
• => screenshot above

[damienmg]

Can you give the reproduction steps?

On Tue, May 30, 2017 at 12:50 AM Hugh Perkins [e-mail]
wrote:

Hi @damienmg https://github.com/damienmg I tried upgrading to bazel
0.5.0, but doesnt seem to work for me for tensorflow?

[image: screen shot 2017-05-30 at 8 48 17 am]
https://cloud.githubusercontent.com/assets/123560/26573285/0909ea7e-4515-11e7-8f86-44d0a02d5fa5.png

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/bazelbuild/bazel/issues/587#issuecomment-304801932,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADjHf_SB-OuA9xTCbsU0LorzaSSpVNgFks5r-8pegaJpZM4GgBZt
.

[mcelhennyi]

I need this to build tensor flow for the native_client in mozilla's DeepSpeech project.

I am behind an authorized proxy, where I need a "domain\user:[email protected]:9090" as my proxy

Even after using (where my $http_host, $https_host, and $prox_port are correct):
bazel --host_jvm_args=-Dhttp.proxyHost=$http_host --host_jvm_args=-Dhttp.proxyPort=$prox_port --host_jvm_args=-Dhttps.proxyHost=$https_host --host_jvm_args=-Dhttps.proxyPort=$prox_port build -c opt --copt=-march=native --copt=-mtune=native --copt=-O3 //tensorflow:libtensorflow_cc.so //native_client:*

I still cannot get past the error:
WARNING: ignoring http_proxy in environment.
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '@io_bazel_rules_closure//closure': Error downloading [http://mirror.bazel.build/github.com/bazelbuild/rules_closure/archive/dec425a4ff3faf09a56c85d082e4eed05d8ce38f.tar.gz, https://github.com/bazelbuild/rules_closure/archive/dec425a4ff3faf09a56c85d082e4eed05d8ce38f.tar.gz] to /home/adc-lab/.cache/bazel/_bazel_adc-lab/4534105d4045448773f96d944be86b59/external/io_bazel_rules_closure/dec425a4ff3faf09a56c85d082e4eed05d8ce38f.tar.gz: All mirrors are down: [java.lang.NullPointerException]
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '@io_bazel_rules_closure//closure': Error downloading [http://mirror.bazel.build/github.com/bazelbuild/rules_closure/archive/dec425a4ff3faf09a56c85d082e4eed05d8ce38f.tar.gz, https://github.com/bazelbuild/rules_closure/archive/dec425a4ff3faf09a56c85d082e4eed05d8ce38f.tar.gz] to /home/adc-lab/.cache/bazel/_bazel_adc-lab/4534105d4045448773f96d944be86b59/external/io_bazel_rules_closure/dec425a4ff3faf09a56c85d082e4eed05d8ce38f.tar.gz: All mirrors are down: [java.lang.NullPointerException]
INFO: Elapsed time: 2.146s
FAILED: Build did NOT complete successfully (0 packages loaded)

Does Auth support domain as well in user name?

[developer-mayuan]

@mcelhennyi I had the same issue as yours.

[JoyChopra1298]

@sunil3590 I tried your method on a CentOS 6 cluster but it isn't working. I downloaded the file on my login node of cluster and created a http server. I tested the server using wget and it is workiing fine. But when I provide the same url to bazel, it cannot download the file.
Note: I am running it using the option --output_user_root=/tmp/bazel since my cluster uses Lustre filesystem which doesn't support bazel.
The command I am running is bazel --output_user_root=/tmp/bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package. I changed the urls in io_bazel_rules_closure in tensorflow workspace.bzl as you mentioned.

[hliang]

Just report that I don't have this issue.
I'm using Centos 7.2, + bazel 0.13.0. Proxy settings for both HTTP_PROXY and HTTPS_PROXY are set (uppercase). Bazel is able to download the packages, although it said "WARNING: ignoring http_proxy in environment." (I read somewhere that the variables are ignored by client, not server - which is doing the downloading).

[JonnoFTW]

I got this working on CentOS, you need to unset http(s)_proxy, after starting a http proxy on host_running_proxy:

ssh -f -N -n -L9999:localhost:9999 host_running_proxy
unset https_proxy
unset http_proxy
export HTTPS_PROXY=http://locahost:9999
export HTTP_PROXY=http://localhost:9999

[chaitanyabandiapps]

Hi everyone, I tried all the options above to try my problem get going, but I'm unable to make it work. I'm using windows 10 on an Azure Deep Learning DSVM, probably the latest build. I'm trying to build the summarize_graph tool using tensorflow's bazel commands.

bazel build graph_transforms:summarize_graph

I'm getting the following error :
tensorflow/core/platform/default/build_config/BUILD:153:1: no such package '@png_archive//'

I'm using Bazel Version "0.15.0".

I tried removing the proxy setting too both using cmd and manually in settings, but that seemed not to work. Could anyone please help me with this ? Please let me know if you need any more information.

[b-ak]

@JonnoFTW That works. Expect that I didn't any proxy server
To elaborate a little.. Here's how it looks like

username@hostname:~$ unset http_proxy 
username@hostname:~$ unset https_proxy 
username@hostname:~$ export HTTPS_PROXY=http://me:[email protected]:8000
username@hostname:~$ export HTTPS_PROXY=http://me:[email protected]:8000

[ackbar03]

I have this issue as well and its really starting to get annoying. It seems that Tensorflow serve with gpu support can only be built with this Bazel thing and the problems existed for 3 years without a solution... They really couldn't have built it some other way : ( , spent my entire day on this.

[ackbar03]

ok so i've finally managed to find a hacky way around this. I'm behind the great firewall of chinois, my vpn doesn't really work anymore and basically my connection keeps getting cut off when ever its trying to fetch something as part of the build process and I have to start again everytime, might be a different problem for those behind a corporate firewall.

basically i run

echo build --experimental_repository_cache ~/.bazel_downloads >>~/.bazelrc

and then instead of running bazel build I run bazel fetch on the target which sort of prefetches it and caches it so you just keep doing that and cursing under your breath until most of the the packages are fetched, and then you run bazel build and hopefully it works.

I'm not actually sure if you need to run echo build --experimental_repository_cache ~/.bazel_downloads >>~/.bazelrc
or it the fetch caches it automatically but I can't be bothered to figure it out. God this was so frustrating, I can't believe theres no easier way to do this

just posting in case it could save someone else the frustration. hhrrrRUAAGHH

[ackbar03]

Does bazel have a mirror or something one can use to download the dependencies from? I had such a bad experience with this last time and I have to do it again tomorrow and it's bringing up some ptsd