Jx: `jx helm step release` returns `403 Unauthorized" (Promote to Environments)

[mikaelmh]

Summary

Steps to reproduce the behavior

1. Install on GKE

2. jx import of existing spring project

Jx version

The output of jx version is:

jx                 1.3.440
jenkins x platform 0.0.2771
Kubernetes cluster v1.9.7-gke.6
kubectl            v1.9.7
helm client        v2.10.0+g9ad53aa
helm server        v2.10.0+g9ad53aa
git                git version 2.14.3 (Apple Git-98)

Kubernetes cluster

GKE, created through jx create

Operating system / Environment

Mac OS

Expected behavior

Actual behavior

[app-name] Running shell script
+ jx step helm release
No $CHART_REPOSITORY defined so using the default value of: http://jenkins-x-chartmuseum:8080
Using helmBinary helm with feature flag: none
Adding missing Helm repo: jenkins-x https://chartmuseum.build.cd.jenkins-x.io
Successfully added Helm repository jenkins-x.
Adding missing Helm repo: releases http://jenkins-x-chartmuseum:8080
Successfully added Helm repository releases.
No $CHART_REPOSITORY defined so using the default value of: http://jenkins-x-chartmuseum:8080
Uploading chart file app-name-1.0.136.tgz to http://jenkins-x-chartmuseum:8080/api/charts
Received 401 response: {"error":"unauthorized"}
error: Failed to post chart to http://jenkins-x-chartmuseum:8080/api/charts due to response 401: {"error":"unauthorized"}
script returned exit code 1

[mikaelmh]

I deleted .jx and .helm folders, deleted the cluster and re-created the cluster a few times now, but still stuck.

Doing some research, it seems the problem might be related https://github.com/chartmuseum/helm-push/issues/11 there not being sufficient permissions given to the pod to make the push. I am wondering if something in the jx installation that has resulted in this. (The issue might be related to #1998 .)

This was working fine before but after upgrading re-installing the cluster fresh, I have this problem. I have tried helm 2.10 and 2.11 but still failing at this step.

@jstrachan any thoughts on approaches I can take. Very grateful for any leads or ideas. Thanks!

[jasine]

same situation on kubernetes 1.10.5, helm version is 2.11

[mikaelmh]

anyone with any progress on this? re-installed from scratch a few times but still stuck on this and not able to get the environments to work.

[danizquierdo18]

same issue, any news?

[kapomaho]

i am also getting this error, i upgraded to 2.11 version, using --no tiller install option nothing works.

[mikaelmh]

We retried this weekend and long story short, we got it to work by removing any special characters from our password.

Background: In Helm 2.10 or 2.11, they seem to have changed the authorization scheme to include Basic Auth. Special characters in basic auth end up being encoded, leading to the new encoded password not being identical to the one stored on jenkinsX. It is a simple thing that we should have caught earlier, but since (a) it did work before the upgrade, and (b) pulling worked (it doesn't require the credentials, see Helm issue in their git repo I referenced earlier ), we did not put 2 and 2 together.

I am closing this issue now, but please feel free to re-open it if your problem persist. My hunch would be that anyone that still encounters should check their auth settings in (~/.jx/chartmuseumAuth.yaml) and in the jenkins console credentials and update accordingly (re-install the cluster after deleting these files, or re-initialize jenkins-x with the new setting)

[shahnewazrifat]

@mikaelmh Can you please guide me how you reset the pass ? We're running Tekton and not Jenkins. Wonder what are the steps you followed.

~/.jx/chartmuseumAuth.yaml is empty like so:
currentserver: "" defaultusername: "" pipelineserver: "" pipelineusername: "" servers: []