Jx: RBAC issue on AKS

[nerumo]

I tried to create an aks cluster but it fails with the problem that rbac isn't activated on AKS. As far as I understood https://github.com/Azure/AKS/issues/67 this something that can't be done right now. How can I create a cluster then with jx?

[jstrachan]

ah damn sorry about that! We'd not got around to testing Jenkins X on AKS since we moved to RBAC (to better handle newer versions of kubernetes which enable it by default). Looks like we need to wait until AKS can support RBAC - sorry about that! We could try figure out how to have a disable RBAC option for Jenkins X as a workaround? I wonder how long until AKS has RBAC?

[jstrachan]

it looks like its aiming for Q1 so am hoping any time now btw!

[herzogf]

I managed to create a k8s cluster via jx in azure/AKS by manually creating the missing ClusterRole BEFORE allowing jx to install the nginx ingress controller. For a detailed error message resulting from this missing ClusterRole see https://github.com/jenkins-x/jx/issues/499#issue-308300141 .

I paused the jx create cluster operation at the following question:
No existing ingress controller found in the kube-system namespace, shall we install one?

And before answering yes I created the missing ClusterRole taken from https://github.com/Azure/acs-engine/issues/1892#issuecomment-353960778 via kubectl create:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: null
  name: cluster-admin
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
rules:
- apiGroups:
  - '*'
  resources:
  - '*'
  verbs:
  - '*'
- nonResourceURLs:
  - '*'
  verbs:
  - '*'

After that helm managed to install the ingress controller.

[rawlingsj]

Awesome - we should be able to add that into the jx create cluster aks command!

[jstrachan]

I figure we may as well check on all installs if there's a clsuter-admin ClusterRole and add it if not? Am sure other clouds might have similar issues

[rawlingsj]

yeah good idea

[tomconte]

Many thanks @herzogf! Just to be complete, if someone wants to deploy on an AKS cluster that was not created with jx create cluster (e.g. created via Azure portal or CLI), one should also run the following command:

kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --ser
viceaccount=kube-system:default

Maybe this is also something jx could test for in the install process?

[rajdavies]

fixed by https://github.com/jenkins-x/jx/pull/612