Jx: upgrade helm to the latest release 2.10.x

[jstrachan]

[jstrachan]

see https://github.com/jenkins-x/builder-base/pull/410

[vunguyenhung]

Hi, [email protected] is not officially released yet and a vast major of users still use version 2.9.1. Even brew still use version 2.9.1 as on https://github.com/Homebrew/homebrew-core/blob/master/Formula/kubernetes-helm.rb#L5.

So I think updating this force users to use the not-so-stable version & will make users can't even run the build if they're using the stable version of helm (which is supposed to work). The error is as follow:

Applying helm chart at . as release name jx-staging to namespace jx-staging
error: failed to run 'helm upgrade --namespace jx-staging --install jx-staging .' command in directory '.', output: 'Error: UPGRADE FAILED: incompatible versions client[v2.10.0-rc.1] server[v2.9.1]
': exit status 1

[ascheman]

I also ran into the same problem as @vunguyenhung on GKE and AWS since I was starting with the local helm as version v2.9.1. I would really appreciate a test in jx if the helm version is correct (as already suggested in #393).

[Gogs008]

I second @ascheman and third @vunguyenhung. I am running into the same issue on AWS EKS. @ascheman, @vunguyenhung, and @jstrachan, what are the work arounds to resolve this issue? I SSH'd into the Jenkins pod in my Kubernetes cluster, and discovered that it is running on a Debian OS, so I tried using apt-get to install a matching version of Helm. However, I was out of luck. Is the next option to SSH in to the pod and run curl https://storage.googleapis.com/kubernetes-helm/helm-v2.10.0-rc.1-linux-amd64.tar.gz | tar xzv && \ mv linux-amd64/helm /usr/bin/ && \ rm -rf linux-amd64
on the Debian OS, or is this too hacky?

Also, on a side note, since I am new to Kubernetes, is it common for my Jenkins pod (or any pod for that matter) to be running just fine, but only intermittently allow me to successfully SSH in, all while being able to log in to the Jenkins console just fine? I have been getting the the following response very often:
ssh: connect to host kubectl port 22: Resource temporarily unavailable while trying to resolve the above mentioned issue.

[vunguyenhung]

@Gogs008 install the latest release of helm here: https://github.com/kubernetes/helm/releases. Jenkins-x automatically installs helm server v2.10.0-rc.1 so you just need to install helm client v2.10.0-rc.1.

[Gogs008]

@vunguyenhung, Thanks for the response! Could you clarify what you mean and exactly what are the steps that need to be done, because if you look at the very last paragraph in the below error output I am encountering in the Jenkins console, it mentions: Error: UPGRADE FAILED: incompatible versions client[v2.10.0-rc.1] server[v2.9.1] ? I read this to mean that it is actually the client that is up to date with [v2.10.0-rc.1], but the server is using [v2.9.1] and needs to be upgraded to [v2.10.0-rc.1]. I figured that "server" means host OS of the Kubernetes pod that Jenkins is sitting in. Could you or anyone for that matter, please clarify my misunderstanding of these terms. Very much appreciated. Thanks and advance!

The error output for my Jenkins Staging pipeline:
`[env] Running shell script

• jx step helm apply
  No $CHART_REPOSITORY defined so using the default value of: http://jenkins-x-chartmuseum:8080
  $HELM_HOME has been configured at /home/jenkins/.helm.
  Not installing Tiller due to 'client-only' flag having been set
  Happy Helming!

Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused
...Successfully got an update from the "releases" chart repository
...Successfully got an update from the "stable" chart repository
...Successfully got an update from the "jenkins-x" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 2 charts
Downloading exposecontroller from repo https://chartmuseum.build.cd.jenkins-x.io
Downloading exposecontroller from repo https://chartmuseum.build.cd.jenkins-x.io
Deleting outdated charts

Applying helm chart at . as release name jx-staging to namespace jx-staging
error: failed to run 'helm upgrade --namespace jx-staging --install jx-staging .' command in directory '.', output: 'Error: UPGRADE FAILED: incompatible versions client[v2.10.0-rc.1] server[v2.9.1]
': exit status 1
script returned exit code 1`

[tdcox]

Running jx version will tell you which helm versions you have on your client machine and the tiller server on your cluster.

The client referred to in the error message is the helm client which is installed in the build container your script is executing in, which has been upgraded recently. If you update your local client and tiller install to match, this will resolve the issue.

[lvthillo]

Also running in this problem on a default minikube cluster creation with jx:
In jenkins I see: FAILED: incompatible versions client[v2.10.0-rc.1] server[v2.9.1]

This while helm version returns

jx version
NAME               VERSION
jx                 1.3.99
jenkins x platform 0.0.1827
kubernetes cluster v1.10.0
kubectl            v1.10.4
helm client        v2.9.1+g20adb27
helm server        v2.9.1+g20adb27
git                git version 2.15.2 (Apple Git-101.1)

[Gogs008]

@tdcox, thanks for your response!

When I run: jx version

it returns:

NAME VERSION
jx 1.3.90
jenkins x platform 0.0.1827
kubernetes cluster v1.10.3
kubectl v1.10.3
helm client v2.9.1+g20adb27
helm server v2.9.1+g20adb27
git git version 2.7.4

A new jx version is available: 1.3.99
? Would you like to upgrade to the new jx version? Yes

Downloading https://github.com/jenkins-x/jx/releases/download/v1.3.99/jx-linux-amd64.tar.gz to /home/{your_name}/.jx/bin/jx.tgz...
Downloaded /home/{yourname}/.jx/bin/jx.tgz

However when I cd over to /home/{your_name}/.jx/bin/ to tar jx.tgz it is nowhere to be found.

Does anyone have any ideas on what are the steps to perform the update/upgrade so I can get jx to version 1.3.99, which I assume will include the new version of the helm client?

I am running Ubuntu in the Linux Subsystem on Windows 10 as my local OS

Thanks again in advance!

[lvthillo]

@Gogs008 I'm using 1.3.99 and it still contains the same issue

[lvthillo]

@Gogs008 What worked for me was downloading manually the newest version (2.10.0-rc1) and extracting it + replacing my older version with it.

$ tar -zxvf helm-v2.10.0-rc.1-darwin-amd64.tar.gz
$ mv darwin-amd64/helm /usr/local/bin/helm
$ helm init --upgrade (don't know if this was necessary)

jx version
NAME               VERSION
jx                 1.3.99
jenkins x platform 0.0.1827
kubernetes cluster v1.10.0
kubectl            v1.10.4
helm client        v2.10.0-rc.1+gaa98e7e
helm server        v2.10.0-rc.1+gaa98e7e
git                git version 2.15.2 (Apple Git-101.1)

[Gogs008]

@lvthillo thanks for that! Very much appreciated! I now have the both versions of the helm client and server matching now!

Does anyone know what appropriate command(s) I need to run so the Jenkins pod is updated and made aware that the Helm Client version has been updated so my builds can finally run? Thanks again in advance!

[lvthillo]

@Gogs008 isn't it not just working now? If you want to update your jenkins platform you can run jx upgrade platform

[Gogs008]

@lvthillo, thanks and I apologize for the late reply (it's been been a crazy day!). Yes it is working now, without me needing to have done anything else! Again, very much appreciated!

[michaelneale]

With a fresh cluster I saw the mismatch error: https://github.com/jenkins-x/jx/issues/1304#issuecomment-405799745

Even with jx downloading the "latest" helm, it was wrong, so this is still a problem for new users who don't know to download and install the rc client on their desktop when installing.

[deyceg]

Think this should be rolled back.

If you look at the 2.10.0-rc.1 release a few things broke my custom helm deployment:

/tiller became /bin/tiller
Tiller runs as nobody which screwed up mounting TLS secrets

Unfortunately I had to dig through the commits on that branch as they don't produce release notes for a candidate version.

[jstrachan]

ok the real 2.10 release is out, so starting the PRs.

After moving to 2.10.rc1 I think we'll miss out future RC releases and just track major releases instead ;)

[qqq978786182]

请问，使用helm v2.10.0版本部署服务，在用sshd服务时，会产生大量sshd的僵尸进程，并且容器内很卡，使用高版本无问题，有人遇到过吗