Bazel: Http Remote Caching GET request doesn't support 301 Redirect starting from 0.11.0

[zimathias]

Description of the problem:

Http Remote Caching doesn't support 301 Redirect anymore.

Bugs: what's the simplest, easiest way to reproduce this bug?

1. Start a bazel remote cache on port 9090:

https://github.com/buchgr/bazel-remote/#prebuilt-image

1. Start a python http redirect server (eg https://gist.github.com/shreddd/b7991ab491384e3c3331):

redirect.py --port 9091 http://localhost:9090

1. Run bazel build configured to use the remote cache on port 9091:

examples/cpp-tutorial/stage1⟫ bazel build --experimental_remote_spawn_cache --remote_http_cache http://localhost:9091 --remote_upload_local_results=false ...
Starting local Bazel server and connecting to it...
.......
INFO: Analysed 2 targets (13 packages loaded).
INFO: Found 2 targets...
WARNING: Some artifacts failed to be downloaded from the remote cache: Download failed with status: 301 Moved Permanently
INFO: Elapsed time: 2.478s, Critical Path: 0.30s
INFO: 4 processes: 2 linux-sandbox, 2 local.
INFO: Build completed successfully, 9 total actions

What operating system are you running Bazel on?

Ubuntu 16.04.

What's the output of bazel info release?

release 0.13.0

Any other information, logs, or outputs that you want to share?

301 Redirect was working properly with bazel 0.10.1 (ie bazel 0.10.1 would properly follow the 301 Redirection). It broke in 0.11.0.

[buchgr]

It was supported unknowingly, as I suppose the apache http library that we used just happened to support it. We can add explicit support for it again. What is your use case for redirects?

[zimathias]

We use an AWS S3 bucket to host the actual cache artifacts. And then use S3 pre signed urls to redirect bazel clients to the S3 bucket to get the actual artifact.

This allows us to run a very simple and small remote cache cluster while leveraging S3 to actually serve all the artifacts.

The remote cache service still needs to handle writing to the S3 bucket. But all cache reads are handled by S3.

[softprops]

I would love to see this get fixed. My company is also trying the s3 signed url redirect approach and are blocked by this

[softprops]

I'm trying to revive this. I've reached back out to the discussion group to seek direction https://groups.google.com/forum/#!topic/bazel-discuss/v9XAyA1LVKo

[buchgr]

I think HTTP redirects are not ideal for GET requests and make no sense for PUT requests.
For GET requests this adds an extra roundtrip. The protocol is very sensitive to roundtrips.
For PUT request we would be transferring one BDP worth of data before receiving the
redirect. So we would be uploading lots of data for no reason. So I would lean towards not
supporting http redirects in the caching protocol as it is hurtful for performance and that's
ultimately the only reason why one uses remote caching.

I think this problem can be better solved by having a simple service that proxies the requests
to S3.

[softprops]

In our case, we actually have something like this, a persistent server that runs proxies requests and writes the data to cache.

The problem we're trying to solve is to reduce the operational cost and maintenance on that. We're looking towards aws lambda and leveraging s3 signed get/put urls do the heavy lifting. The redirect roundtrip in our case was something we'd were conscious of but for us this was an okay tradeoff for decreasing the operational cost of running a persistent server to support our remote cache.

[nikhedonia]

Any updates on this issue?
I've just tested the latest commit on master and it still doesn't seem to handle redirects.

For GET requests this adds an extra roundtrip

Redirecting to the closest and fastest server, especially for bigger artefacts is worth the extra roundtrip.
Furthermore it reduces the overhead of running a proxy service.

[buchgr]

@nikhedonia I am happy to accept a patch that handles redirects and documentation on the implications. I currently don't have the resources to implement it myself.