Hi
By default,k3s components' certificates is 1 year,those components include:kube-apiserver,scheduler,cloud-controller,K3s should rotate those certificates.
K3s version: v1.17.4+k3s1
A clear and concise description of what you want to happen.
The components' certificates expire,k3s will not work.
Thanks.
Hi
It bothers me, too.
Do you have a good way to update the certificate automatically?|
Thanks.
Certificates rotate automatically if they are <90 days from expiration when k3s is started. This has been the case since v0.10: https://github.com/rancher/k3s/pull/805
As long as you are patching and restarting your k3s infrastructure at least every couple months, you shouldn't run into expired certificates.
@brandond
Does it mean we have to restart k3s if we want to update certificate?
Thanks.
Yes. Make sure you're running at least 0.10, and restart k3s. If the certs are less than 90 days from expiration they will be rotated when it starts. I don't believe there's a way to manually rotate them or make them valid for longer... other than poking at the files manually with a 3rd party tool like openssl of course.
You're restarting your k3s nodes at least every couple months to apply OS patches, right?
@brandond
OK Thanks.
Since the question has been answered, maybe the issue can be closed. cc @liyongxian .
Closing the issue. If there are further questions let us know and we can reopen if necessary.
Hi @brandond ,
I did a test on VM, modify system time to the day before expired time ( < 90 days) and restart k3s, then checking cert info by using openssl command as following, found the cert has no change.
openssl s_client -connect localhost:6443 -showcerts < /dev/null 2>&1 | openssl x509 -noout -enddate
I tried reset the time several times even clean up the certs in /var/lib/rancher/k3s/server/tls, but unlucky, k3s unable to start up. It seems that somewhere cached the certificates?
Jun 2 17:15:06 localhost k3s: E0602 17:15:06.593592 7212 reflector.go:153] k8s.io/[email protected]/tools/cache/reflector.go:105: Failed to list *v1.VolumeAttachment: Get "https://127.0.0.1:6444/apis/storage.k8s.io/v1/volumeattachments?limit=500&resourceVersion=0": x509: certificate has expired or is not yet valid: current time 2020-06-02T17:15:06Z is before 2022-04-26T23:02:59Z
Could you please advise if there is any misunderstanding?
Version: v1.17.5+k3s1
Thanks,
@ethinx did you set the time into the future, start k3s, and then set it back again? According to the error, your cert was generated almost two years in the future and is not yet valid:
x509: certificate has expired or is not yet valid: current time 2020-06-02T17:15:06Z is before 2022-04-26T23:02:59Z
@brandond since I tried to clean up the certs under /var/lib/rancher/server/tls and screw things up.
Make it simpler, I did a fresh install and test again as following, do I miss something here?
# INSTALL_K3S_SKIP_DOWNLOAD=true ./install.sh
# openssl s_client -connect localhost:6443 -showcerts </dev/null 2>&1 | openssl x509 -noout -startdate -enddate
notBefore=Jun 3 13:27:43 2020 GMT
notAfter=Jun 3 13:27:43 2021 GMT
[root@localhost ~]# date -s 20210520
Thu May 20 00:00:00 UTC 2021
[root@localhost ~]# hwclock -w
[root@localhost ~]# date
Thu May 20 00:00:08 UTC 2021
# systemctl restart k3s
# openssl s_client -connect localhost:6443 -showcerts </dev/null 2>&1 | openssl x509 -noout -startdate -enddate
notBefore=Jun 3 13:27:43 2020 GMT
notAfter=Jun 3 13:27:43 2021 GMT
# k3s -v
k3s version v1.17.5+k3s1 (58ebdb2a)
@ethinx Don't set the time forward while it's running. If you're going to test it that way, stop k3s before changing the system time. Also, make sure you've disabled any time sync daemons - ntpd, chrony, systemd-timesyncd, etc.
Hi @brandond,
It seems that not about the time sync daemons, even though I stop chronyd (I didn't notice it before), cert keep unchanged after I stop k3s -> set time to future -> start k3s.
I think I find where the cert be cached, it's the k3s-serving secret under namespace kube-system.
The cert get updated after I remove the secret and local cache file dynamic-cert.json. You could check the procedures in my gist: https://gist.github.com/ethinx/c0ede41a7a6a340681476ef564ea75f2
Just have a glance on the code, seems that the local file certificate generation is different to the secret.
Hope you could share more about it, as we don't have documents about this.
I think you might be correct. @galal-hussein @erikwilson it looks like the kube-apiserver, kubelet, and auth-proxy certs all call through certutil.LoadOrGenerateKeyFile which handles rotation, but the dynamiclistener/factory stuff that creates the k3s-serving key has its own CA loader and cert generation code that does not appear to check expiration or handle renewal.
Reopening the issue based on comments it seems like we may want to investigate this further.
Hi there,
Following the Rancher 2.X installation instructions for Rancher on K3s, in this document: https://rancher.com/docs/rancher/v2.x/en/installation/k8s-install/kubernetes-rke/, we get to copying the kubectl config to a local machine to try to connect and we receive the following error:
Unable to connect to the server: x509: certificate has expired or is not yet valid: current time 2020-06-24T14:40:08-05:00 is after 2017-10-24T21:09:52Z
Any thoughts?
Thanks!
@ieugen please close it.I did it by changing k3s code and rebuild it. Thanks.
@liyongxian I can't close it since I did not start the thread.
You should be able to close it or someone from the project.
It would be nice if you can share a link to the steps you used to solve it.
Certificates are rotated if they are < 90 days from expiration.
Before expiration:
client-admin.crt
notAfter=Jul 13 19:22:39 2021 GMT
client-auth-proxy.crt
notAfter=Jul 13 19:22:39 2021 GMT
client-ca.crt
notAfter=Jul 11 19:22:39 2030 GMT
client-cloud-controller.crt
notAfter=Jul 13 19:22:39 2021 GMT
client-controller.crt
notAfter=Jul 13 19:22:39 2021 GMT
client-k3s-controller.crt
notAfter=Jul 13 19:22:39 2021 GMT
client-kube-apiserver.crt
notAfter=Jul 13 19:22:39 2021 GMT
client-kube-proxy.crt
notAfter=Jul 13 19:22:39 2021 GMT
client-scheduler.crt
notAfter=Jul 13 19:22:39 2021 GMT
request-header-ca.crt
notAfter=Jul 11 19:22:39 2030 GMT
server-ca.crt
notAfter=Jul 11 19:22:39 2030 GMT
serving-kube-apiserver.crt
notAfter=Jul 13 19:22:39 2021 GMT
Update date
date -s 20210514
Confirm certs are rotated
client-admin.crt
notAfter=May 14 00:00:14 2022 GMT
client-auth-proxy.crt
notAfter=May 14 00:00:14 2022 GMT
client-ca.crt
notAfter=Jul 11 19:22:39 2030 GMT
client-cloud-controller.crt
notAfter=May 14 00:00:14 2022 GMT
client-controller.crt
notAfter=May 14 00:00:14 2022 GMT
client-k3s-controller.crt
notAfter=May 14 00:00:14 2022 GMT
client-kube-apiserver.crt
notAfter=May 14 00:00:14 2022 GMT
client-kube-proxy.crt
notAfter=May 14 00:00:14 2022 GMT
client-scheduler.crt
notAfter=May 14 00:00:14 2022 GMT
request-header-ca.crt
notAfter=Jul 11 19:22:39 2030 GMT
server-ca.crt
notAfter=Jul 11 19:22:39 2030 GMT
serving-kube-apiserver.crt
notAfter=May 14 00:00:14 2022 GMT
Further testing: If the system date is set to after exipiration date, rotation works but certificate are not updated. Since the certificates being rotated by default within <90 of expiration should solve the issue.
Suppose the expiration date is
notAfter=Jul 13 21:29:36 2021 GMT
and set the date to < 90 days to expire.
date -s 20210514
systemctl restart k3s
Certs are rotated and updated
-> notAfter=May 14 00:00:19 2022 GMT
kubectl get nodes
NAME STATUS ROLES AGE VERSION
ip-172-31-2-249 Ready master 304d v1.17.4+k3s1
Suppose date is set to after expire
date -s 20220516
Certificates are rotated but not updated
kubectl get nodes
Unable to connect to the server: x509: certificate has expired or is not yet valid
systemctl restart k3s
-> notAfter=May 16 00:00:38 2023 GMT
kubectl get nodes
Unable to connect to the server: x509: certificate has expired or is not yet valid
@ShylajaDevadiga can you also show the output of the k3-serving cert for each of those two tests:
kubectl get secret -n kube-system k3s-serving -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text | grep Not
After rotation before <90 of expiration
# date -s 20210514
Fri May 14 00:00:00 UTC 2021
# systemctl restart k3s
# for i in `ls *.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done
client-admin.crt
notAfter=May 14 00:00:32 2022 GMT
client-auth-proxy.crt
notAfter=May 14 00:00:32 2022 GMT
client-ca.crt
notAfter=Jul 11 22:03:48 2030 GMT
client-cloud-controller.crt
notAfter=May 14 00:00:32 2022 GMT
client-controller.crt
notAfter=May 14 00:00:32 2022 GMT
client-k3s-controller.crt
notAfter=May 14 00:00:32 2022 GMT
client-kube-apiserver.crt
notAfter=May 14 00:00:32 2022 GMT
client-kube-proxy.crt
notAfter=May 14 00:00:32 2022 GMT
client-scheduler.crt
notAfter=May 14 00:00:32 2022 GMT
request-header-ca.crt
notAfter=Jul 11 22:03:48 2030 GMT
server-ca.crt
notAfter=Jul 11 22:03:48 2030 GMT
serving-kube-apiserver.crt
notAfter=May 14 00:00:32 2022 GMT
# kubectl get secret -n kube-system k3s-serving -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text | grep Not
Not Before: Jul 13 22:03:48 2020 GMT
Not After : Jul 13 22:03:48 2021 GMT
#
After expiration
# date -s 20220614
Tue Jun 14 00:00:00 UTC 2022
# systemctl restart k3s
# for i in `ls *.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done
client-admin.crt
notAfter=Jun 14 00:00:04 2023 GMT
client-auth-proxy.crt
notAfter=Jun 14 00:00:04 2023 GMT
client-ca.crt
notAfter=Jul 11 22:03:48 2030 GMT
client-cloud-controller.crt
notAfter=Jun 14 00:00:04 2023 GMT
client-controller.crt
notAfter=Jun 14 00:00:04 2023 GMT
client-k3s-controller.crt
notAfter=Jun 14 00:00:04 2023 GMT
client-kube-apiserver.crt
notAfter=Jun 14 00:00:04 2023 GMT
client-kube-proxy.crt
notAfter=Jun 14 00:00:04 2023 GMT
client-scheduler.crt
notAfter=Jun 14 00:00:04 2023 GMT
request-header-ca.crt
notAfter=Jul 11 22:03:48 2030 GMT
server-ca.crt
notAfter=Jul 11 22:03:48 2030 GMT
serving-kube-apiserver.crt
notAfter=Jun 14 00:00:04 2023 GMT
# kubectl get secret -n kube-system k3s-serving -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text | grep Not
Unable to connect to the server: x509: certificate has expired or is not yet valid
unable to load certificate
140525255820608:error:0909006C:PEM routines:get_name:no start line:../crypto/pem/pem_lib.c:745:Expecting: TRUSTED CERTIFICATE
root@ip-172-31-0-203:/var/lib/rancher/k3s/server/tls#
Oh right, kubectl will of course not connect once it's expired. Just to verify the dates you can do:
echo QUIT | openssl s_client -showcerts -connect localhost:6443 | openssl x509 -noout -text | grep Not
@ShylajaDevadiga can you retest this with c8282f4939ea8e36b5411936ed174787eaefaccc
@brandond using k3s commit k3s-serving secret cert does not seem to be rotated.
Befor rotation:
k3s -v
k3s version v1.18.6+k3s-c8282f49 (c8282f49)
for i in `ls *.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done
client-admin.crt
notAfter=Aug 5 01:15:04 2021 GMT
client-auth-proxy.crt
notAfter=Aug 5 01:15:04 2021 GMT
client-ca.crt
kubectl get secret -n kube-system k3s-serving -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text | grep Not
Not Before: Aug 5 01:15:04 2020 GMT
Not After : Aug 5 01:15:04 2021 GMT
After rotation before <90 days
for i in `ls *.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done
client-admin.crt
notAfter=Jun 6 00:05:40 2022 GMT
client-auth-proxy.crt
notAfter=Jun 6 00:05:40 2022 GMT
kubectl get secret -n kube-system k3s-serving -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text | grep Not
Not Before: Aug 5 01:15:04 2020 GMT
Not After : Aug 5 01:15:04 2021 GMT
After setting the date to past expiration date, none of the certs got rotated
date -s 20220806
for i in `ls *.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done
client-admin.crt
notAfter=Jun 6 00:05:40 2022 GMT
client-auth-proxy.crt
notAfter=Jun 6 00:05:40 2022 GMT
echo QUIT | openssl s_client -showcerts -connect localhost:6443 | openssl x509 -noout -text | grep Not
Can't use SSL_get_servername
depth=0 O = k3s, CN = k3s
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 O = k3s, CN = k3s
verify error:num=21:unable to verify the first certificate
verify return:1
DONE
Not Before: Aug 5 01:15:04 2020 GMT
Not After : Aug 5 01:15:04 2021 GMT
not sure is it related to this issue
how do I do to renew these certificates manually ?
k3s service is now looping crash on boot ...
update
I rename these certs into .old postfix, and k3s just renew them in server side
k3s servicer is able to boot, but error logs still not change
Aug 05 04:57:52 k3s-cluster k3s[8937]: W0805 04:57:52.011531 8937 authorization.go:47] Authorization is disabled
Aug 05 04:57:52 k3s-cluster k3s[8937]: W0805 04:57:52.011558 8937 authentication.go:55] Authentication is disabled
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.108263 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.121091 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.121471 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.121750 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.122013 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.122260 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.122513 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.122746 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.123003 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.123244 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.123521 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.123982 8937 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
Aug 05 04:57:52 k3s-cluster k3s[8937]: E0805 04:57:52.131172 8937 reflector.go:126] k8s.io/client-go/informers/factory.go:130: Failed to list *v1.PersistentVolumeClaim: Unauthorized
Aug 05 04:57:52 k3s-cluster k3s[8937]: time="2020-08-05T04:57:52.131643821Z" level=fatal msg="starting tls server: Unauthorized"
Aug 05 04:57:52 k3s-cluster systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Aug 05 04:57:52 k3s-cluster systemd[1]: k3s.service: Failed with result 'exit-code'.
Aug 05 04:57:52 k3s-cluster systemd[1]: Failed to start Lightweight Kubernetes.
# for i in `ls *.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done
client-admin.crt
notAfter=Jul 27 08:48:39 2020 GMT
client-auth-proxy.crt
notAfter=Jul 27 08:48:39 2020 GMT
client-ca.crt
notAfter=Jul 25 08:48:39 2029 GMT
client-controller.crt
notAfter=Jul 27 08:48:39 2020 GMT
client-kube-apiserver.crt
notAfter=Jul 27 22:53:14 2021 GMT
client-kube-proxy.crt
notAfter=Jul 27 08:48:39 2020 GMT
client-scheduler.crt
notAfter=Jul 27 08:48:39 2020 GMT
request-header-ca.crt
notAfter=Jul 25 08:48:39 2029 GMT
server-ca.crt
notAfter=Jul 25 08:48:39 2029 GMT
serving-kube-apiserver.crt
notAfter=Jul 27 23:00:49 2021 GMT
@CasparChou deleting or renaming all your certs doesn't force them to renew, instead k3s generates new ones. This causes all kinds of other problems and is not a good idea. If there is a problem you are trying to solve, please open a new issue. All certs except a single apiserver certificate should rotate automatically on recent releases of k3s.
@ShylajaDevadiga was k3s stopped while you changed the date? I haven't looked into the logic but I would expect it to only check expiration on startup.
@CasparChou deleting or renaming all your certs doesn't force them to renew, instead k3s generates new ones. This causes all kinds of other problems and is not a good idea. If there is a problem you are trying to solve, please open a new issue. All certs except a single apiserver certificate should rotate automatically on recent releases of k3s.
thanks, I think I need to upgrade it first.
@ShylajaDevadiga it appears that the cert expiration check sleeps for 5 minutes at startup, and then only checks every 6 hours afterwards: https://github.com/rancher/k3s/blob/c8282f4939ea8e36b5411936ed174787eaefaccc/vendor/github.com/rancher/dynamiclistener/listener.go#L155
This means that the test should involve:
I added some extra info-level printfs to a local build, and it appears to be checking properly:
Jul 1 00:01:12 centos01 k3s: time="2021-07-01T00:01:12.397685015-07:00" level=info msg="checking certificate: days=90"
Jul 1 00:01:12 centos01 k3s: time="2021-07-01T00:01:12.398319926-07:00" level=info msg="certificate expiration: is 2021-09-29 07:01:12.39831146 +0000 UTC after 2021-08-04 19:55:45 +0000 UTC"
Jul 1 00:01:12 centos01 k3s: time="2021-07-01T00:01:12.398337639-07:00" level=info msg="refreshing certificate"
Jul 1 00:01:12 centos01 k3s: time="2021-07-01T00:01:12.401464944-07:00" level=info msg="Active TLS secret k3s-serving (ver=174) (count 15): map[listener.cattle.io/cn-10.0.1.137:10.0.1.137 listener.cattle.io/cn-10.0.1.158:10.0.1.158 listener.cattle.io/cn-10.0.1.160:10.0.1.160 listener.cattle.io/cn-10.0.1.162:10.0.1.162 listener.cattle.io/cn-10.0.1.170:10.0.1.170 listener.cattle.io/cn-10.43.0.1:10.43.0.1 listener.cattle.io/cn-127.0.0.1:127.0.0.1 listener.cattle.io/cn-BARBAZ:BARBAZ listener.cattle.io/cn-FOOBAR:FOOBAR listener.cattle.io/cn-TEST01:TEST01 listener.cattle.io/cn-TEST02:TEST02 listener.cattle.io/cn-kubernetes:kubernetes listener.cattle.io/cn-kubernetes.default:kubernetes.default listener.cattle.io/cn-kubernetes.default.svc.cluster.local:kubernetes.default.svc.cluster.local listener.cattle.io/cn-localhost:localhost listener.cattle.io/hash:f601ed9e6448891d133d41278c5498aab5ca2062b9003b72afe09fc7549b65ae]"
The final line (Active TLS secret k3s-serving) is not something I added - you should see this in the master build when the cert is updated.
However despite it refreshing, it does not seem to actually update the NotBefore/NotAfter dates on the cert. I'll go figure out why this isn't working.
@brandond Thanks. Found the same after including a wait. k3s-serving cert does not get rotated.
OK, so it looks like checkExpiration calls Refresh but that just calls AddCN which returns early because NeedsUpdate returns false because the SANs are not changing. We need to pass through a flag to force reissuance even if the SANs aren't changing.
Verified using K3s commit a2471a1f8a2aa26902f8e3b29624dc9c809024d2. k3s-serving cert rotation succeeded.
Before rotation
root@ip-172-31-10-200:/var/lib/rancher/k3s/server/tls# for i in `ls *.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done
client-admin.crt
notAfter=Aug 20 08:32:18 2021 GMT
client-auth-proxy.crt
notAfter=Aug 20 08:32:18 2021 GMT
client-ca.crt
notAfter=Aug 18 08:32:18 2030 GMT
client-cloud-controller.crt
notAfter=Aug 20 08:32:18 2021 GMT
client-controller.crt
notAfter=Aug 20 08:32:18 2021 GMT
client-k3s-controller.crt
notAfter=Aug 20 08:32:18 2021 GMT
client-kube-apiserver.crt
notAfter=Aug 20 08:32:18 2021 GMT
client-kube-proxy.crt
notAfter=Aug 20 08:32:18 2021 GMT
client-scheduler.crt
notAfter=Aug 20 08:32:18 2021 GMT
request-header-ca.crt
notAfter=Aug 18 08:32:18 2030 GMT
server-ca.crt
notAfter=Aug 18 08:32:18 2030 GMT
serving-kube-apiserver.crt
notAfter=Aug 20 08:32:18 2021 GMT
root@ip-172-31-10-200:/var/lib/rancher/k3s/server/tls# kubectl get secret -n kube-system k3s-serving -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text | grep Not
Not Before: Aug 20 08:32:18 2020 GMT
Not After : Aug 20 08:32:18 2021 GMT
Certs are rotated when date is manually set to <90 to expiration
date -s 20210710
root@ip-172-31-10-200:/var/lib/rancher/k3s/server/tls# for i in `ls *.crt`; do echo $i; openssl x509 -enddate -noout -in $i; done
client-admin.crt
notAfter=Jul 10 00:00:10 2022 GMT
client-auth-proxy.crt
notAfter=Jul 10 00:00:10 2022 GMT
client-ca.crt
notAfter=Aug 18 08:32:18 2030 GMT
client-cloud-controller.crt
notAfter=Jul 10 00:00:10 2022 GMT
client-controller.crt
notAfter=Jul 10 00:00:10 2022 GMT
client-k3s-controller.crt
notAfter=Jul 10 00:00:10 2022 GMT
client-kube-apiserver.crt
notAfter=Jul 10 00:00:10 2022 GMT
client-kube-proxy.crt
notAfter=Jul 10 00:00:10 2022 GMT
client-scheduler.crt
notAfter=Jul 10 00:00:10 2022 GMT
request-header-ca.crt
notAfter=Aug 18 08:32:18 2030 GMT
server-ca.crt
notAfter=Aug 18 08:32:18 2030 GMT
serving-kube-apiserver.crt
notAfter=Jul 10 00:00:10 2022 GMT
root@ip-172-31-10-200:/var/lib/rancher/k3s/server/tls# kubectl get secret -n kube-system k3s-serving -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -text | grep Not
Not Before: Aug 20 08:32:18 2020 GMT
Not After : Jul 10 00:00:40 2022 GMT