Installer: console 403: no RBAC policy matched for system:anonymous
https://github.com/openshift/installer/blob/master/docs/dev/libvirt-howto.md#connect-to-the-cluster-console
https://${OPENSHIFT_INSTALL_CLUSTER_NAME}-api.${OPENSHIFT_INSTALL_BASE_DOMAIN}:6443/console/
does not pull up a console, just json output.
bf36c903c58c385315fa74aa9eb216dca89a797a
michaelgugino
All 31 comments
does not pull up a console, just json output.
What is the JSON output? Things have been exciting since #330 landed, there are many things that could be going wrong ;).
Also, we have a nice issue template... ;). Even if you push your issues with hub or whatever instead of going through GitHub's web UI, those are the sort of things we're looking for in bug reports.
wking
on 3 Oct 2018
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/console\": no RBAC policy matched",
"reason": "Forbidden",
"details": {
},
"code": 403
}
Yeah, it wasn't working last week either, not sure if the output was the same, but never got a console.
michaelgugino
on 3 Oct 2018
"message": "forbidden: User \"system:anonymous\" cannot get path \"/console\": no RBAC policy matched",
We think this is openshift/origin#20983.
wking
on 3 Oct 2018
Ran into the same problem.
Platform: libvirt
Accessing: https://test1-api.tt.testing:6443/console
Response:
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/console\": no RBAC policy matched",
"reason": "Forbidden",
"details": {
},
"code": 403
}
$ bin/openshift-install version
bin/openshift-install v0.1.0-65-gc0331204949479b41010ff7553297075a777bc2a
Terraform v0.11.8
$ oc get pods --all-namespaces [±master ●]
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system kube-apiserver-f2lzx 1/1 Running 0 33m
kube-system kube-controller-manager-b6985978d-z8tnw 1/1 Running 0 33m
kube-system kube-core-operator-7f4d6b8dcf-6kzgl 1/1 Running 0 26m
kube-system kube-dns-787c975867-p8dtz 3/3 Running 0 33m
kube-system kube-flannel-5rn9f 2/2 Running 0 27m
kube-system kube-flannel-fdv56 2/2 Running 0 27m
kube-system kube-flannel-z7vcb 2/2 Running 4 19m
kube-system kube-proxy-kbs8t 1/1 Running 0 33m
kube-system kube-proxy-klfqx 1/1 Running 0 19m
kube-system kube-proxy-lkwf8 1/1 Running 0 33m
kube-system kube-scheduler-78d86f9754-gdkdd 1/1 Running 0 33m
kube-system metrics-server-5767bfc576-tzp4p 2/2 Running 0 24m
kube-system pod-checkpointer-5847n 1/1 Running 0 33m
kube-system pod-checkpointer-5847n-test1-master-0 1/1 Running 0 32m
kube-system tectonic-network-operator-qbgn7 1/1 Running 0 33m
openshift-cluster-api clusterapi-apiserver-6b855f7bc5-bxqmp 2/2 Running 0 26m
openshift-cluster-api clusterapi-controllers-85f6bfd9d5-96cd6 2/2 Running 0 25m
openshift-cluster-api machine-api-operator-5d85454676-8tgmf 1/1 Running 0 33m
openshift-cluster-version bootstrap-cluster-version-operator-test1-bootstrap 1/1 Running 1 33m
openshift-cluster-version cluster-version-operator-8bb6cff75-nkwrl 1/1 Running 0 33m
openshift-core-operators openshift-service-cert-signer-operator-6dbfb84555-wsm54 1/1 Running 0 6m
openshift-ingress default-http-backend-6985d557bb-r5w29 1/1 Running 0 25m
openshift-ingress router-67b6b747c5-jp7vk 1/1 Running 0 25m
openshift-ingress tectonic-ingress-controller-operator-fcb9c6f4b-jtdcn 1/1 Running 0 26m
openshift-machine-config-operator machine-config-controller-5b74549d47-w6q65 1/1 Running 0 26m
openshift-machine-config-operator machine-config-daemon-jfjrz 1/1 Running 3 19m
openshift-machine-config-operator machine-config-daemon-mfjvb 1/1 Running 0 24m
openshift-machine-config-operator machine-config-operator-68fcdcb4f9-7bxqd 1/1 Running 0 33m
openshift-machine-config-operator machine-config-server-8rprn 1/1 Running 0 25m
openshift-operator-lifecycle-manager catalog-operator-7db987f97b-xdrcj 0/1 CreateContainerError 0 33m
openshift-operator-lifecycle-manager olm-operator-748f55cdd-m254c 0/1 CreateContainerError 0 33m
openshift-operator-lifecycle-manager olm-operator-867dbbb8f9-sj9dk 0/1 CreateContainerError 0 5m
openshift-operator-lifecycle-manager package-server-7fb4c799bc-v4h4c 0/1 CreateContainerError 0 33m
openshift-service-cert-signer apiservice-cabundle-injector-7bc57bccf4-hrz9z 1/1 Running 0 5m
openshift-service-cert-signer configmap-cabundle-injector-5889f997bc-4cxmn 1/1 Running 0 5m
openshift-service-cert-signer service-serving-cert-signer-5d867cb55d-44gxm 1/1 Running 0 5m
tectonic-system kube-addon-operator-784b4b6c7-d8sjh 1/1 Running 0 26m
Was the web-console replaced by something?
mtnbikenc
on 9 Oct 2018
I was getting same json issue and just run below command
oc login -u system:admin
Now i am able to access it
imranrazakhan
on 18 Oct 2018
Same error I am getting today.
# kubectl get --all-namespaces pods
NAMESPACE NAME READY STATUS RESTARTS AGE
default registry-6fcb8b7789-4x72w 1/1 Running 0 20m
kube-system kube-apiserver-krqjc 1/1 Running 0 41m
kube-system kube-controller-manager-b6985978d-zxhm2 1/1 Running 0 41m
kube-system kube-core-operator-7f4d6b8dcf-hdfgw 1/1 Running 0 32m
kube-system kube-dns-787c975867-kz8np 3/3 Running 0 41m
kube-system kube-flannel-h2kqm 2/2 Running 5 29m
kube-system kube-flannel-lt5dd 2/2 Running 0 36m
kube-system kube-flannel-t4l49 2/2 Running 0 36m
kube-system kube-proxy-5xz66 1/1 Running 0 29m
kube-system kube-proxy-kvgxr 1/1 Running 0 41m
kube-system kube-proxy-nbqvd 1/1 Running 0 41m
kube-system kube-scheduler-78d86f9754-j4pr8 1/1 Running 0 41m
kube-system metrics-server-5767bfc576-67znk 2/2 Running 0 30m
kube-system pod-checkpointer-7v2xn 1/1 Running 0 41m
kube-system pod-checkpointer-7v2xn-test1-master-0 1/1 Running 0 39m
kube-system tectonic-network-operator-76qt7 1/1 Running 0 41m
openshift-apiserver apiserver-kgcgc 1/1 Running 0 31m
openshift-cluster-api clusterapi-apiserver-6b855f7bc5-s2flm 2/2 Running 0 33m
openshift-cluster-api clusterapi-controllers-85f6bfd9d5-vrw2s 2/2 Running 0 31m
openshift-cluster-api machine-api-operator-5d85454676-m9z9d 1/1 Running 0 39m
openshift-cluster-version cluster-version-operator-fqkl6 1/1 Running 0 41m
openshift-controller-manager controller-manager-w9kdb 1/1 Running 0 30m
openshift-core-operators openshift-cluster-openshift-apiserver-operator-5fbd49d8f7-vlthq 1/1 Running 0 39m
openshift-core-operators openshift-cluster-openshift-controller-manager-operator-7cw26h6 1/1 Running 0 39m
openshift-core-operators openshift-service-cert-signer-operator-6d6c6f55db-jspwh 1/1 Running 0 39m
openshift-image-registry cluster-image-registry-operator-58c7c9bfd6-r7l92 1/1 Running 0 29m
openshift-ingress tectonic-ingress-controller-operator-fcb9c6f4b-tlhzq 0/1 CrashLoopBackOff 9 31m
openshift-machine-config-operator machine-config-controller-6948b45dd9-n8cxd 1/1 Running 0 32m
openshift-machine-config-operator machine-config-daemon-drsmd 1/1 Running 4 29m
openshift-machine-config-operator machine-config-daemon-fsxft 1/1 Running 0 31m
openshift-machine-config-operator machine-config-operator-545fcb447d-h9pfj 1/1 Running 0 39m
openshift-machine-config-operator machine-config-server-5dwbh 1/1 Running 0 31m
openshift-monitoring cluster-monitoring-operator-c64f5b475-tlplc 1/1 Running 0 29m
openshift-monitoring prometheus-operator-5bf8644c75-xmrcm 1/1 Running 0 20m
openshift-operator-lifecycle-manager catalog-operator-5d5d8c7689-wf2ql 1/1 Running 0 39m
openshift-operator-lifecycle-manager olm-operator-76b7f57649-8zj5v 1/1 Running 0 39m
openshift-operator-lifecycle-manager package-server-f994b8699-f7hsc 0/1 CrashLoopBackOff 8 39m
openshift-service-cert-signer apiservice-cabundle-injector-d4c746869-z66ks 1/1 Running 0 32m
openshift-service-cert-signer configmap-cabundle-injector-77bd46b-bqnd8 1/1 Running 0 32m
openshift-service-cert-signer service-serving-cert-signer-55fb7cc589-l6rzs 1/1 Running 0 32m
openshift-web-console webconsole-86f4f55644-cl9r5 1/1 Running 0 20m
tectonic-system kube-addon-operator-784b4b6c7-fdkqj 1/1 Running 0 32m
== web console output
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/console/\": no RBAC policy matched",
"reason": "Forbidden",
"details": {
},
"code": 403
}
@imranrazakhan where did you run that oc command to login?
praveenkumar
on 18 Oct 2018
@praveenkumar i just run this command on shell of VM where i installed 3.11
[root@c1-ocp ~]# oc login -u system:admin
imranrazakhan
on 18 Oct 2018
This is the old console that is being removed. The new console is behind a route. You're seeing this because the master console proxy is removed (since it's no longer needed).
I believe @deads2k was going to add a redirect to the new console from the old URL, but I'm not sure if that's in place.
spadgett
on 22 Oct 2018
We think this is openshift/origin#20983.
It's a different cause.
spadgett
on 22 Oct 2018
I believe @deads2k was going to add a redirect to the new console from the old URL, but I'm not sure if that's in place.
Do you know the new URL? A redirect would be nice, but we may be able to help a bunch of people by updating our docs.
wking
on 22 Oct 2018
For new 4.0 installs, it is https://console-openshift-console.<default-routing-suffix>
cc @benjaminapetersen
spadgett
on 22 Oct 2018
@spadgett I'm not entirely sure what you mean by default-routing-suffix. There does not seem to be anything at the following location:
# curl https://console.openshift-console.${OPENSHIFT_INSTALL_BASE_DOMAIN}
curl: (6) Could not resolve host: console.openshift-console.tt.testing
edit: this is calling it from the KVM host, not KVM guest
stlaz
on 25 Oct 2018
@stlaz Look at the route in the openshift-console namespace. You can see the hostname there. By default, console uses a generated hostname.
spadgett
on 25 Oct 2018
Thanks. Here's what I did:
# oc get route -n openshift-console
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
console console-openshift-console.router.default.svc.cluster.local console https reencrypt/Redirect None
Curling this just gives me Could not resolve hots:. I wonder which service is used to add this to the KVM host machine DNS? I am using the dnsmasq with NetworkManager solution from the guide. For the record, the following works just fine with it:
# host $OPENSHIFT_INSTALL_CLUSTER_NAME-bootstrap.$OPENSHIFT_INSTALL_BASE_DOMAIN
test1-bootstrap.tt.testing has address 192.168.126.10
Yeah, we depend on the default route working. I don't think it does out of the box yet.
You could add the console hostname to the /etc/hosts or update the Console CR to another hostname. (@benjaminapetersen, does the operator currently reconcile console hostname?)
spadgett
on 25 Oct 2018
The console-operator does not yet reconcile a custom hostname from its cr.yaml. Still some discussion about how that should actually work.
benjaminapetersen
on 1 Nov 2018
Routing workaround for AWS from @ironcladlou: https://gist.github.com/ironcladlou/784e4bd5cdd7e270ae0bea444809cbfd If anyone goes through something similar for libvirt, please post your notes here.
wking
on 14 Nov 2018
Anyone knows how to enable console for the libvirt?
jianzhangbjz
on 20 Nov 2018
See this comment: https://gist.github.com/ironcladlou/784e4bd5cdd7e270ae0bea444809cbfd#gistcomment-2764321
ATM, libvert seems to be the trickiest option. There are workarounds for AWS & GCE around.
benjaminapetersen
on 20 Nov 2018
There is now a 'special user' who goes by the name kubeadmin configured. (12/5/18 this was enabled)
To access the web-console (aws):
- install with latest installer
- after install completes export kubeconfig, wait a few minutes (or keep obsessively trying to
oc login -u kubeadmin -p 5char-5char-5char-5charuntil it works - the password is logged at end of install, also it is written to
${INSTALL_DIR}/auth/kubeadmin-password - grab the route from -n openshift-console, you might get an error at first when you try to access it,
https://console-openshift-console.apps.clustername.devcluster.openshift.com, if you dooc delete pod/openshift-console-blahblah -n openshift-consolethen wait a minute and try again - enter username
kubeadminand password5char-5char-5char-5charto login to console
(you can now also view prometheus/grafana console)
sallyom
on 5 Dec 2018
grab the route from -n openshift-console, you might get an error at first when you try to access it
This problem is fixed, so you shouldn't need to delete the pod anymore. Console should just work now on AWS. (We just need to log the URL, #782)
spadgett
on 6 Dec 2018
@sallyom So https://github.com/openshift/installer/issues/411#issuecomment-444620427 will only for AWS side, from libvirt there is still no way to get the console?
praveenkumar
on 6 Dec 2018
@spadgett confirmed don't need the delete pod hack today (although even after logging in to the console, going to prometheus and using Login with OpenShift gives me a login screen).
RobertKrawitz
on 6 Dec 2018
To access the console from libvirt, execute the following steps:
Allow kubectl to bind to privileged ports:
$ sudo setcap CAP_NET_BIND_SERVICE=+eip /usr/bin/kubectl
Note: If you omit the above, you have to start kubectl using sudo. Next, port-forward the router-default service:
$ kubectl -n openshift-ingress port-forward svc/router-internal-default 443
Get the routes and add bind them to 127.0.0.1 in /etc/hosts:
$ kubectl get routes --all-namespaces
NAMESPACE NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
openshift-console console console-openshift-console.apps.test1.tt.testing console https reencrypt/Redirect None
openshift-monitoring grafana grafana-openshift-monitoring.apps.test1.tt.testing grafana https reencrypt None
openshift-monitoring prometheus-k8s prometheus-k8s-openshift-monitoring.apps.test1.tt.testing prometheus-k8s web reencrypt None
$ cat /etc/hosts
127.0.0.1 console-openshift-console.apps.test1.tt.testing
127.0.0.1 grafana-openshift-monitoring.apps.test1.tt.testing
127.0.0.1 prometheus-k8s-openshift-monitoring.apps.test1.tt.testing
Use the credentials as described in https://github.com/openshift/installer/issues/411#issuecomment-444620427
Open i.e. https://grafana-openshift-monitoring.apps.test1.tt.testing or https://console-openshift-console.apps.test1.tt.testing.
s-urbaniak
on 7 Dec 2018
I just tried @s-urbaniak suggestion and it works as said. Thanks for this quick workaround :+1:
praveenkumar
on 7 Dec 2018
To access the console from libvirt, execute the following steps:
Allow
kubectlto bind to privileged ports:$ sudo setcap CAP_NET_BIND_SERVICE=+eip /usr/bin/kubectlNote: If you omit the above, you have to start kubectl using
sudo. Next, port-forward therouter-defaultservice:$ kubectl -n openshift-ingress port-forward svc/router-default 443Get the routes and add bind them to
127.0.0.1in/etc/hosts:$ kubectl get routes --all-namespaces NAMESPACE NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD openshift-console console console-openshift-console.apps.test1.tt.testing console https reencrypt/Redirect None openshift-monitoring grafana grafana-openshift-monitoring.apps.test1.tt.testing grafana https reencrypt None openshift-monitoring prometheus-k8s prometheus-k8s-openshift-monitoring.apps.test1.tt.testing prometheus-k8s web reencrypt None $ cat /etc/hosts 127.0.0.1 console-openshift-console.apps.test1.tt.testing 127.0.0.1 grafana-openshift-monitoring.apps.test1.tt.testing 127.0.0.1 prometheus-k8s-openshift-monitoring.apps.test1.tt.testingUse the credentials as described in #411 (comment)
Open i.e. https://grafana-openshift-monitoring.apps.test1.tt.testing or https://console-openshift-console.apps.test1.tt.testing.
I have to mention that if the libvirt machine have no additional memory or no X to open a browser, then we have to use proxy to make the browser and terminal work on another machine with something like https://gist.github.com/corehello/6f50472d9b4f9a5624c49312c17d3e99
And setup necessary proxy env for terminal and also browser:
✗ cat proxy.sh
export HTTPS_PROXY=http://10.66.140.70:8888/
export ALL_PROXY=socks://10.66.140.70:8888/
export ftp_proxy=http://10.66.140.70:8888/
export FTP_PROXY=http://10.66.140.70:8888/
export no_proxy=localhost,127.0.0.0/8,::1
export https_proxy=http://10.66.140.70:8888/
export HTTP_PROXY=http://10.66.140.70:8888/
export all_proxy=socks://10.66.140.70:8888/
export http_proxy=http://10.66.140.70:8888/
export NO_PROXY=localhost,127.0.0.0/8,::1
wjiangjay
on 11 Dec 2018
Since this is working out of the box now for AWS and the incorrect instructions were removed from the README, closing the issue.
/close
spadgett
on 8 Jan 2019
@spadgett: Closing this issue.
In response to this:
Since this is working out of the box now for AWS and the incorrect instructions were removed from the README, closing the issue.
/close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
openshift-ci-robot
on 8 Jan 2019
@s-urbaniak Since most of the folks are using you suggested way https://github.com/openshift/installer/issues/411#issuecomment-445165262 to get the console and recently svc name is changed from router-default to router-internal-default , can you update that this comment with?
$ kubectl -n openshift-ingress port-forward svc/router-internal-default 443
Just a quick note, and I haven't had time to dig into the background of the use case, but it's unsafe to rely on resource names like router-internal-default. We make no guarantees about that service name, or even the existence of the service — it should be considered an implementation detail. Of course when it comes to libvirt right now a working hack is probably going to proliferate anyway, but I noticed this and wanted to drop a disclaimer 😁
ironcladlou
on 23 Feb 2019
@praveenkumar i just run this command on shell of VM where i installed 3.11
[root@c1-ocp ~]# oc login -u system:admin
DONE..! Thank You,,!
nileshalhat
on 11 Jun 2019
Related issues
alexouzounis
·
12Comments
davivcgarcia
·
5Comments
kikisdeliveryservice
·
11Comments
glennodickson
·
5Comments
jamesregis
·
4Comments
Most helpful comment
To access the console from libvirt, execute the following steps:
Allow
kubectlto bind to privileged ports:Note: If you omit the above, you have to start kubectl using
sudo. Next, port-forward therouter-defaultservice:Get the routes and add bind them to
127.0.0.1in/etc/hosts:Use the credentials as described in https://github.com/openshift/installer/issues/411#issuecomment-444620427
Open i.e. https://grafana-openshift-monitoring.apps.test1.tt.testing or https://console-openshift-console.apps.test1.tt.testing.