After redeploying certificates, I get a 502 error when trying to access the web console.
The other APIs look fine and /healthz/ready responds ok
Version
oc v3.9.0+191fece
kubernetes v1.9.1+a0ce1bc657
features: Basic-Auth
openshift v3.9.0+a96a520-22
kubernetes v1.9.1+a0ce1bc657
Steps To Reproduce
- Launch a redeploy certificates
- Try to access the web console
Current Result
502 error when accessing https://openshift/console
Expected Result
Acess to WebConsole
Additional Information
During certificate redeploy, an error occured and I had to restart dnsmasq on the first master. It seems that because of that problem some weird configuration occured. I'm trying now to understand what happened.
After investigatin I'm arriving at that situation:
# oc get service webconsole -n openshift-web-console
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
webconsole ClusterIP 172.30.100.254 <none> 443/TCP 27d
From within the Openshift master API container:
# curl -vk https://172.30.100.254/console/
* About to connect() to 172.30.100.254 port 443 (#0)
* Trying 172.30.100.254...
* Connected to 172.30.100.254 (172.30.100.254) port 443 (#0)
...
> GET /console/ HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 172.30.100.254
> Accept: */*
>
< HTTP/1.1 200 OK
< Cache-Control: no-cache, no-store
< Referrer-Policy: strict-origin-when-cross-origin
< Vary: Accept-Encoding
< X-Content-Type-Options: nosniff
< X-Dns-Prefetch-Control: off
< X-Frame-Options: DENY
< X-Xss-Protection: 1; mode=block
< Date: Thu, 14 Jun 2018 13:43:46 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
<
<!doctype html>
<html class="no-js layout-pf layout-pf-fixed">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=EDGE"/>
<meta charset="utf-8">
<base href="/console/">
...
So from what I can see: the Webconsole pods are running ok and are on the master nodes. From within the API container, I'm able to access the clusterIP of the service. But when the Origin-master-API tries to proxy, then it gets a 502 error:
# curl -vk https://localhost/console/
* About to connect() to localhost port 443 (#0)
* Trying ::1...
* Connection refused
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* skipping SSL peer certificate verification
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
* Server certificate:
* subject: CN=10.252.6.72
* start date: Jun 14 01:24:24 2018 GMT
* expire date: Jun 13 01:24:25 2020 GMT
* common name: 10.252.6.72
* issuer: CN=openshift-signer@1526615121
> GET /console/ HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost
> Accept: */*
>
< HTTP/1.1 502 Bad Gateway
< Date: Thu, 14 Jun 2018 14:31:38 GMT
< Content-Length: 0
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host localhost left intact
dmetzler
All 9 comments
@openshift/sig-master @openshift/sig-user-interface
enj
on 14 Jun 2018
I just ran into the same issue. Seems like the ansible playbook does not recreate the secret/webconsole-serving-cert from the webconsole, which is a autogenerated service serving certificate.
From the docs it should be sufficient to delete the secret to regenerate it, but for some reason this didn't work for me, I got it running by:
oc delete secret webconsole-serving-cert
oc delete svc/webconsole
and finally running the ansible playbook for the webconsole
ansible-playbook -i inventory/hosts.localhost playbooks/openshift-web-console/config.yml
kadhonn
on 15 Jun 2018
Thank you that worked. Instead of re-executing the playbook I did the following:
oc delete secret webconsole-serving-cert
oc delete pods webconsole=true
After deleting the secret, it is automatically recreated by the service. You then have to restart all pods to take the newly generated secret
dmetzler
on 15 Jun 2018
why did you close this issue? it still is a problem with the ansible redeploy-certs-script, we just have a workaround
kadhonn
on 16 Jun 2018
I'll open a new ticket on the ansible repo, this one is not done for that purpose.
dmetzler
on 16 Jun 2018
ahh, ok, thanks :)
kadhonn
on 17 Jun 2018
Thank you that worked. Instead of re-executing the playbook I did the following:
oc delete secret webconsole-serving-cert oc delete pods webconsole=trueAfter deleting the secret, it is automatically recreated by the service. You then have to restart all pods to take the newly generated secret
This worked for me in OpenShift 3.11 to fix the cluster console, the regular webconsole was working.
oc project openshift-console
oc delete secret console-serving-cert
oc delete pods console*
cragr
on 6 Nov 2018
i application console not working , try fix but not working
and restart master node ,fix.
oc v3.11.0+62803d0-1
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO
Server https://master195:8443
openshift v3.11.0+06cfa24-67
kubernetes v1.11.0+d4cacc0
pangzheng
on 22 Mar 2019
@cragr thank you!!! worked for me. I dont know whats happened with console on my cluster.
Nurlan199206
on 22 Sep 2020
Related issues
davivd
路
4Comments
edseymour
路
3Comments
alikhajeh1
路
3Comments
surajssd
路
4Comments
guangxuli
路
4Comments
Most helpful comment
Thank you that worked. Instead of re-executing the playbook I did the following:
After deleting the secret, it is automatically recreated by the service. You then have to restart all pods to take the newly generated secret