Openj9: Helm Chart: Hosting binary helm chart on Artifactory

[chrisc66]

This issue is originally opened in the internal infrastructure repo, also opening the discussion here as we are open for additional discussion or suggestions.

As a continuous part of the OpenJ9 Helm Chart discussion, we are looking for a location (possibly on Artifactory) to host binary helm chart in tgz file format.

The binary helm chart is used by the helm CLI for downloading and deploying the chart directly to a cluster. Once the binary helm charts are uploaded to the server, we can provide the download URL (REPO_URL) to helm.

$ helm repo add REPO_NAME REPO_URL
"REPO_NAME" has been added to your repositories

$ helm repo list
NAME        URL
REPO_NAME   REPO_URL

Requirements from the server

• Needs to be externally accessible by direct download and helm CLI.
• The directory needs to follow below structure so that helm can recognize the directory as a helm chart repo.

$ tree repo
repo
├── index.yaml
└── openj9-jitserver-chart-0.1.0.tgz

References

• Link to helm chart issue: https://github.com/eclipse/openj9/issues/9443
• Link to helm chart PR: https://github.com/eclipse/openj9-utils/pull/8

[pshipton]

The first suggestion is to use the existing OpenJ9 artifactory, which is used to temporarily store build/test (ci) artifacts. https://140-211-168-230-openstack.osuosl.org/artifactory/ci-eclipse-openj9/ already exists, but it doesn't seem appropriate to put the helm chart under ci-eclipse-openj9 so another top level directory should be used.

[AdamBrousseau]

Does the chart need to be stored as a binary tarball? It appears to be a few small text files. Can we not check those in to the repo directly? If a binary is needed, what amount of download traffic will we have for it? Is this for development purposes or is it part of our release? If the latter, I would rather not host "product" on our dev artifact server. Github can host binaries (outside of the repo) so that may be a better option.

[chrisc66]

Does the chart need to be stored as a binary tarball? It appears to be a few small text files. Can we not check those in to the repo directly?

Yes, the binaries are needed. The binaries are called packaged helm chart, which is simply zipping the helm chart (yaml
and other text files) into one tgz file. The tgz file is required by helm CLI, as helm cannot recognize helm chart (yaml files) directly. Another reason for having packaged helm chart is for version tracking, a packaged helm chart is similar to a code release.

e.g. helm install checks whether binary helm charts are available in helm repositories on its record. helm install docs

Is this for development purposes or is it part of our release? If the latter, I would rather not host "product" on our dev artifact server.

Hosting binary helm chart should be included in the release as it is part of the helm chart. As mentioned in the helm repo docs, "you can use a Google Cloud Storage (GCS) bucket, Amazon S3 bucket, GitHub Pages, or even create your own web server".

Maybe we could upload binary files to another branch other than master (gh-pages or a separate branch) so that there wont be overheads when pulling from master? Not aware of other ways to host binaries on Github but outside of repo, could you point it to me?

[jdekonin]

I think what you want is more like what is posted for releases, maybe even just adding a helm chart binary to each release?
https://github.com/eclipse/openj9/releases

@pshipton do you have this process documented?

[pshipton]

https://github.com/eclipse/openj9/releases is a github feature. It's just a bunch of text, I don't think it can be used to post the binary helm chart, but it can contain the link to the chart once we decide where to put it.

[AdamBrousseau]

https://docs.github.com/en/free-pro-team@latest/github/administering-a-repository/managing-releases-in-a-repository

1. Optionally, to include binary files such as compiled programs in your release, drag and drop or manually select files in the binaries box.

[pshipton]

Note the helm chart is (at least) two files, which need to exist side by side.

<link>/index.yaml
<link>/openj9-jitserver-chart-<ver>.tgz

I think adding a branch to the openj9-utils repo would work. We put the content into the branch and use https://raw.githubusercontent.com/eclipse/openj9-utils/<branch>/ as the root link.

[pshipton]

I guess this layout isn't strictly required: "It is not required that a chart package be located on the same server as the index.yaml file. However, doing so is often the easiest."

[pshipton]

Using github releases to store the binary, we'd get a link such as https://github.com/eclipse/openj9/files/5782068/openj9-jitserver-chart-<ver>.tgz

[AdamBrousseau]

Note that putting it in a different branch does not alleviate the initial concerns of "don't store binary files in Git". IMO if this is a utils repo and the binaries are going to stay small it's probably fine to leave them in the Git repo.

[pshipton]

I expect anybody can create a github link for the chart, by adding the binary to any issue. Once the link is created, the index.yaml can be updated, and the same link added to the release page (if desired).

[chrisc66]

Apologies if I wasn't clear on the directory requirement. We also want to add new helm chart binaries continuously to the same location. The initial idea is to have a server that stores a directory structure (repo) that we can upload additional files later on. The chart version will follow openj9 release version, so something like:

$ tree repo
repo
├── index.yaml
├── openj9-jitserver-chart-0.24.0.tgz
├── openj9-jitserver-chart-0.24.1.tgz
├── openj9-jitserver-chart-0.25.0tgz
├── ...
└── openj9-jitserver-chart-x.x.x.tgz

The helm repo URL provided to helm CLI should be an URL to repo. The helm CLI will choose the proper chart version for users based on index.yaml, which basically records chart name, versions, and paths to binaries.

I guess this layout isn't strictly required: "It is not required that a chart package be located on the same server as the index.yaml file. However, doing so is often the easiest."

Could we try including index.yaml in the same directory as binaries? This index.yaml keeps track of the version so that users only need to give a name (and optionally a version) to the helm CLI once the helm repository (URL to repo) is set. If we are not going to provide a URL to a specific binary chart instead of helm repo, users will have to keep tracks of versioning.

Note that putting it in a different branch does not alleviate the initial concerns of "don't store binary files in Git". IMO if this is a utils repo and the binaries are going to stay small it's probably fine to leave them in the Git repo.

The size of a binary chart is relatively small. It only contains around 10 YAML files and the size is less than 10KB. Not sure if this is acceptable in the utils repo if we store them in a separate branch?

[AdamBrousseau]

The size of a binary chart is relatively small. It only contains around 10 YAML files and the size is less than 10KB. Not sure if this is acceptable in the utils repo if we store them in a separate branch?

The separate branch doesn't do anything other than put it on another branch. When you clone the repo by default you pull all the meta for the entire repo. However, as I said, in my opinion, given the size of the binaries, and the fact that they won't change but will only be more added (correct me if I'm wrong) I think putting them in the repo is fine. The concerns around storing binaries in Git comes from bloating the repo with large objects that can change but cannot be diffed. Adding a few KB 6 times a year is nothing.

[pshipton]

Using links like https://github.com/eclipse/openj9-utils/raw/<branch>/<folder>/<file> seem to work. I created a mockup branch in my fork.

https://github.com/pshipton/openj9-utils/raw/helmcharts/releases/index.yaml
https://github.com/pshipton/openj9-utils/raw/helmcharts/releases/openj9-jitserver-chart-0.24.0.tgz
https://github.com/pshipton/openj9-utils/raw/helmcharts/releases/openj9-jitserver-chart-0.24.1.tgz

[pshipton]

See also https://github.com/pshipton/openj9-utils/tree/helmcharts/releases

[chrisc66]

Sounds great, thanks. I will create a separate PR into openj9-utils for adding branch and hosting binary files.

[DanHeidinga]

I have a strong, almost allergic, reaction against putting binaries into source control. And the problems always begin in this way - let's check in this one, little, never gonna change binary. And then it snowballs into more and more binaries. And they need to be changed because a file had to be updated, and on and on.

Let's avoid starting down this path.

Based on the info in this issue, it seems like we can put the helm chart's index.yaml in our repo and host the binary zips on the github releases (which do support uploading loading binaries). Then the index.yaml can point to the release binaries with no need to commit the binary to our repo.

Are there any deal breakers with this approach? The "it's easier" reason to check them into the repo is not a sufficient reason to proceed with that approach.

[AdamBrousseau]

I'm fine with using GitHub releases for hosting if it suits our requirements. My main concern was using the development Artifactory server which is not intended for production release hosting and has no real guaranteed uptime or support agreement.

[chrisc66]

Thanks, @DanHeidinga and @AdamBrousseau. I understand your concerns, and I have double-checked that hosting binary charts in releases is a valid solution in our case. The PR has been updated https://github.com/eclipse/openj9-utils/pull/8#issuecomment-759928378.

[pshipton]

To be clear, the binaries will be hosted by attaching them to any Issue (or PR), which results in a link to the binary, like https://github.com/eclipse/openj9-utils/files/<some number>/<attachment name>. Attach them by dragging and dropping on the Issue (or PR) to get a link.

There is nothing special about attaching them to a release page, and I don't want any dependency on the release page for creating the links. The release page is just text like any Issue, and can contain links. I expect the release page can refer to the index.yml, but doesn't need to refer to each helm chart binary. The index.yml will be updated via PR, but the links to the binaries can be created in advance (by attaching them to an Issue/PR) and approved as part of the PR review process.

[chrisc66]

Thanks for the suggestion, it looks to me that attaching the binaries to a PR makes more sense.

For instance, if there is a version upgrade PR for the helm chart, a binary will be uploaded to that PR and the URL will be updated in the same PR as well.

Edit:

Unfortunately, the binary helm chart is in tgz format, which is not supported for upload in GitHub issue / PR. The question comes back to the possibility of hosting binary charts on the release page.

[pshipton]

How about tar.gz? The release page doesn't support anything different from Issues/PRs.

test-images.tar.gz

[chrisc66]

Thanks, the binary helm chart will be stored as a file in PRs.

Closing this issue since no further discussion needed.