Elasticsearch: Use AdoptOpenJDK API to Download JDKs

[james-crowley]

Currently, when Elasticsearch bundles a JDK with a release it comes from the Oracle’s website or from a selfhosted Artifactory server hosted by Elastic depending on the bundled_jdk configs in version.properties.

Not sure the design decision behind using Artifactory but AdoptOpenJDK now provides an official API to retrieve downloads.

When picking AdoptOpenJDK as a vendor, Elasticsearch should use the API to retrieve JDKs from the official AdoptOpenJDK source.

Here are the advantages:

• Unifying the process to download JDKs. Both, AdoptOpenJDK and Oracle, will now pull from their respective official download sources

• People will not have to trust “random” binaries on Elastic’s Artifactory server

• Elastic no longer needs to maintain an external antifactory for JDKs

[james-crowley]

Thank you to @johnoliver and @smlambert for your help on this issue and providing the PR https://github.com/AdoptOpenJDK/openjdk-api-v3/pull/168.

[elasticmachine]

Pinging @elastic/es-core-infra (:Core/Infra/Packaging)

[rjernst]

@james-crowley The artifactory repository we use serves 3 purposes. First, it is a consistent, simple url pattern that we can control. In the past we have been burned by the url pattern changing on us (as can be seen in the special cases in JdkDownloadPlugin). Second, it is a caching layer, where just our builds (in theory at least) consume this URL. We have had a lot of trouble with external services wreaking havoc in our CI since even a few minutes hiccup can propagate to dozens of failed builds that must be investigated. Third, when we originally started using adoptopenjdk, the download links redirected to github, which did not support HEAD requests, and this broke our use of pulling the artifact through Gradle's dependency management.

Internally, we have a "jvm catalog" that is used within CI, and it is our intention to eventually use that, either directly or indirectly (some jvm licenses make exposing them, even just for testing, tricky!). While using this api in the interim would be possible, we don't have much confidence in the stability of the API due to both the newness of it (only 3 months old) and the quick creation and deprecations of v1 and v2. Given that, at this time we don't plan to make this change, so I hope you don't mind that I close this issue.

[mark-vieira]

I'm reopening this as we've run into issues with folks on less stable internet connections downloading these JDKs from Artifactory. I suspect using the AdoptOpenJDK downloads will be superior as they likely leverage a CDN which helps our distributed contributors.

[jasontedor]

I suspect using the AdoptOpenJDK downloads will be superior as they likely leverage a CDN which helps our distributed contributors.

It's not that they likely do, we know they do because the initial API call merely redirects to GitHub:

curl -v https://api.adoptopenjdk.net/v3/binary/version/jdk-14+36/mac/x64/jdk/hotspot/normal/adoptopenjdk
*   Trying 104.17.158.60...
[...]
> GET /v3/binary/version/jdk-14+36/mac/x64/jdk/hotspot/normal/adoptopenjdk HTTP/2
> Host: api.adoptopenjdk.net
> User-Agent: curl/7.64.1
> Accept: */*
> 
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 307 
< date: Thu, 16 Apr 2020 16:02:18 GMT
< content-length: 0
< set-cookie: __cfduid=dfaf62d32fa3de9cbd9c11cc26faf4f811587052938; expires=Sat, 16-May-20 16:02:18 GMT; path=/; domain=.adoptopenjdk.net; HttpOnly; SameSite=Lax
< access-control-allow-origin: *
< location: https://github.com/AdoptOpenJDK/openjdk14-binaries/releases/download/jdk-14%2B36/OpenJDK14U-jdk_x64_mac_hotspot_14_36.tar.gz
[...]

[mark-vieira]

the initial API call merely redirects to GitHub:

Why is it we are not running into the HEAD request issue here then?

[jasontedor]

Why is it we are not running into the HEAD request issue here then?

I'm wondering the same thing and actively trying to figure that out.

[jasontedor]

It looks like GitHub added support for HEAD requests now, as far as I can tell, this was done in collaboration with Gradle, presumably so that Gradle could use GitHub artifact repositories. See #5322 (comment).

[james-crowley]

Third, when we originally started using adoptopenjdk, the download links redirected to github, which did not support HEAD requests, and this broke our use of pulling the artifact through Gradle's dependency management.

Yup, I saw the same issue with my initial testing. Once I saw the issue, I reached out to the API team at AdoptOpenJDK to see if they would be willing to support a HEAD request as GitHub does not support it. That PR got merged into the API a last week, https://github.com/AdoptOpenJDK/openjdk-api-v3/pull/168.

@mark-vieira and @jasontedor if you take a look at that PR I linked, the Adopt API was modified to allow Gradle to do HEAD requests against the Adopt API.

[jasontedor]

I think GitHub supports them directly now:

$ curl -I https://github.com/AdoptOpenJDK/openjdk14-binaries/releases/download/jdk-14%2B36/OpenJDK14U-jdk_x64_mac_hotspot_14_36.tar.gz
HTTP/1.1 302 Found
server: GitHub.com
date: Thu, 16 Apr 2020 16:09:46 GMT
content-type: text/html; charset=utf-8
status: 302 Found
vary: X-PJAX, Accept-Encoding, Accept, X-Requested-With
location: https://github-production-release-asset-2e65be.s3.amazonaws.com/233878254/ccd39400-6911-11ea-8e6b-4f0ba3a561d9?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200416%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200416T160946Z&X-Amz-Expires=300&X-Amz-Signature=17916229cfd5f76b4df044d75c7f0b97a0163a235dde0eca85913aef1f174b18&X-Amz-SignedHeaders=host&actor_id=0&repo_id=233878254&response-content-disposition=attachment%3B%20filename%3DOpenJDK14U-jdk_x64_mac_hotspot_14_36.tar.gz&response-content-type=application%2Foctet-stream
[...]

[james-crowley]

From my testing, it looks like Gradle follows that redirect to the aws s3 location, which does not allow for HEAD requests. It returns a 403 Forbidden.

curl -I https://github-production-release-asset-2e65be.s3.amazonaws.com/233878254/ccd39400-6911-11ea-8e6b-4f0ba3a561d9\?X-Amz-Algorithm\=AWS4-HMAC-SHA256\&X-Amz-Credential\=AKIAIWNJYAX4CSVEH53A%2F20200416%2Fus-east-1%2Fs3%2Faws4_request\&X-Amz-Date\=20200416T160946Z\&X-Amz-Expires\=300\&X-Amz-Signature\=17916229cfd5f76b4df044d75c7f0b97a0163a235dde0eca85913aef1f174b18\&X-Amz-SignedHeaders\=host\&actor_id\=0\&repo_id\=233878254\&response-content-disposition\=attachment%3B%20filename%3DOpenJDK14U-jdk_x64_mac_hotspot_14_36.tar.gz\&response-content-type\=application%2Foctet-stream
HTTP/1.1 403 Forbidden
x-amz-request-id: A19BEE3B324C0225
x-amz-id-2: AUJqOll9ZZzm4JJGeJysF25rdqZHYBW090p6nTwu+y2Akz2zyxENEUM5CS+OWhAvRzR7aTsHRtQ=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Thu, 16 Apr 2020 16:30:21 GMT
Server: AmazonS3

[mark-vieira]

@james-crowley I believe the 403 you are getting is due to the redirect location being a one-time use URL.

[james-crowley]

@mark-vieira and @rjernst both you mentioned a jvm catalog, it sounds pretty interesting can either of you elaborate more on what this is? If that is the way Elastic will be moving in the future, I would love to get more information, so I could help out in any way I can.

Also @mark-vieira you mentioned on the PR, when you guys get a HA solution for Artifactory that you might want to convert back to using Artifactory. Did you want me to change up the PR to allow for three download options? The three being Oracle's downloads, Adopt's API and Elastic's Artifactory server? That way you guys would not need to convert back and people would have the choice of those three to download their JDKs.

[mark-vieira]

@mark-vieira and @rjernst both you mentioned a jvm catalog, it sounds pretty interesting can either of you elaborate more on what this is? If that is the way Elastic will be moving in the future, I would love to get more information, so I could help out in any way I can.

Right now its purely internal. We eventually want to open it up but there are issues around distributing JDKs with less permissive licenses. We'll need to sort that out before integrating it into our open source projects.

Did you want me to change up the PR to allow for three download options?

No need. We'll cross that bridge if and when we come to it.