Rke: RKE Installation error ETCD Health Check Problem
RKE version: 0.2.5
Operating system and kernel: centos 7
Type/provider of hosts: openstack
Steps to Reproduce:
I am tyring to run rke -d up command. After some point, it gives health check error for etcd cluster. Error is like following:
INFO[0060] [etcd] Successfully started etcd plane.. Checking etcd cluster health
DEBU[0060] [etcd] Check etcd cluster health
DEBU[0071] Failed to get /health for host [10.154.12.89]: Get https://172.16.100.40:2379/health: net/http: TLS handshake timeout
DEBU[0086] Failed to get /health for host [10.154.12.89]: Get https://172.16.100.40:2379/health: net/http: TLS handshake timeout
DEBU[0102] Failed to get /health for host [10.154.12.89]: Get https://172.16.100.40:2379/health: net/http: TLS handshake timeout
DEBU[0107] [etcd] Check etcd cluster health
DEBU[0117] Failed to get /health for host [10.154.12.79]: Get https://172.16.100.15:2379/health: net/http: TLS handshake timeout
DEBU[0133] Failed to get /health for host [10.154.12.79]: Get https://172.16.100.15:2379/health: net/http: TLS handshake timeout
DEBU[0148] Failed to get /health for host [10.154.12.79]: Get https://172.16.100.15:2379/health: net/http: TLS handshake timeout
DEBU[0153] [etcd] Check etcd cluster health
DEBU[0164] Failed to get /health for host [10.154.12.78]: Get https://172.16.100.28:2379/health: net/http: TLS handshake timeout
DEBU[0179] Failed to get /health for host [10.154.12.78]: Get https://172.16.100.28:2379/health: net/http: TLS handshake timeout
DEBU[0195] Failed to get /health for host [10.154.12.78]: Get https://172.16.100.28:2379/health: net/http: TLS handshake timeout
FATA[0200] [etcd] Failed to bring up Etcd Plane: [etcd] Etcd Cluster is not healthy
I also checked control node for etcd container logs and I found following:
2019-07-18 11:25:00.708898 I | etcdmain: rejected connection from "172.16.100.40:43876" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.728766 I | etcdmain: rejected connection from "172.16.100.28:44722" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.784241 I | etcdmain: rejected connection from "172.16.100.28:44726" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.796296 I | etcdmain: rejected connection from "172.16.100.40:43882" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.815147 I | etcdmain: rejected connection from "172.16.100.40:43884" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.821242 I | raft: d17ee7c963f68089 is starting a new election at term 374
2019-07-18 11:25:00.821261 I | raft: d17ee7c963f68089 became candidate at term 375
2019-07-18 11:25:00.821269 I | raft: d17ee7c963f68089 received MsgVoteResp from d17ee7c963f68089 at term 375
2019-07-18 11:25:00.821276 I | raft: d17ee7c963f68089 [logterm: 1, index: 3] sent MsgVote request to d9c6974366ce1ba at term 375
2019-07-18 11:25:00.821282 I | raft: d17ee7c963f68089 [logterm: 1, index: 3] sent MsgVote request to 9019c714a7a7fe3b at term 375
2019-07-18 11:25:00.833387 I | etcdmain: rejected connection from "172.16.100.28:44730" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.888661 I | etcdmain: rejected connection from "172.16.100.28:44734" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.900891 I | etcdmain: rejected connection from "172.16.100.40:43890" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.919489 I | etcdmain: rejected connection from "172.16.100.40:43892" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.939689 I | etcdmain: rejected connection from "172.16.100.28:44738" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:00.993042 I | etcdmain: rejected connection from "172.16.100.28:44742" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.005607 I | etcdmain: rejected connection from "172.16.100.40:43898" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.024015 I | etcdmain: rejected connection from "172.16.100.40:43900" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.045844 I | etcdmain: rejected connection from "172.16.100.28:44746" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.097482 I | etcdmain: rejected connection from "172.16.100.28:44750" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.111659 I | etcdmain: rejected connection from "172.16.100.40:43906" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.130189 I | etcdmain: rejected connection from "172.16.100.40:43908" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.155660 I | etcdmain: rejected connection from "172.16.100.28:44754" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.203604 I | etcdmain: rejected connection from "172.16.100.28:44758" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.217830 I | etcdmain: rejected connection from "172.16.100.40:43914" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.236230 I | etcdmain: rejected connection from "172.16.100.40:43916" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.256765 I | etcdmain: rejected connection from "172.16.100.28:44762" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.309802 I | etcdmain: rejected connection from "172.16.100.28:44766" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.324055 I | etcdmain: rejected connection from "172.16.100.40:43922" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.342295 I | etcdmain: rejected connection from "172.16.100.40:43924" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.361748 I | etcdmain: rejected connection from "172.16.100.28:44770" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.414243 I | etcdmain: rejected connection from "172.16.100.28:44774" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.428648 I | etcdmain: rejected connection from "172.16.100.40:43930" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.446695 I | etcdmain: rejected connection from "172.16.100.40:43932" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.466458 I | etcdmain: rejected connection from "172.16.100.28:44778" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.518936 I | etcdmain: rejected connection from "172.16.100.28:44782" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.533229 I | etcdmain: rejected connection from "172.16.100.40:43938" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.551178 I | etcdmain: rejected connection from "172.16.100.40:43940" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.572610 I | etcdmain: rejected connection from "172.16.100.28:44786" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.623518 I | etcdmain: rejected connection from "172.16.100.28:44790" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.637605 I | etcdmain: rejected connection from "172.16.100.40:43946" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.655754 I | etcdmain: rejected connection from "172.16.100.40:43948" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.677373 I | etcdmain: rejected connection from "172.16.100.28:44794" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.729742 I | etcdmain: rejected connection from "172.16.100.28:44798" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.743776 I | etcdmain: rejected connection from "172.16.100.40:43954" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.761868 I | etcdmain: rejected connection from "172.16.100.40:43956" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.783646 I | etcdmain: rejected connection from "172.16.100.28:44802" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.835898 I | etcdmain: rejected connection from "172.16.100.28:44806" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.849959 I | etcdmain: rejected connection from "172.16.100.40:43962" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.867992 I | etcdmain: rejected connection from "172.16.100.40:43964" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.889989 I | etcdmain: rejected connection from "172.16.100.28:44810" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.940220 I | etcdmain: rejected connection from "172.16.100.28:44816" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.947005 I | etcdmain: rejected connection from "172.16.100.28:44818" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.954343 I | etcdmain: rejected connection from "172.16.100.40:43970" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.972617 I | etcdmain: rejected connection from "172.16.100.40:43972" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:01.999263 I | etcdmain: rejected connection from "172.16.100.28:44822" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.046431 I | etcdmain: rejected connection from "172.16.100.28:44826" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.060888 I | etcdmain: rejected connection from "172.16.100.40:43978" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.079187 I | etcdmain: rejected connection from "172.16.100.40:43980" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.103844 I | etcdmain: rejected connection from "172.16.100.28:44830" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.152498 I | etcdmain: rejected connection from "172.16.100.28:44834" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.166856 I | etcdmain: rejected connection from "172.16.100.40:43986" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.183693 I | etcdmain: rejected connection from "172.16.100.40:43988" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.209979 I | etcdmain: rejected connection from "172.16.100.28:44838" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.258653 I | etcdmain: rejected connection from "172.16.100.28:44842" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.273014 I | etcdmain: rejected connection from "172.16.100.40:43994" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.289726 I | etcdmain: rejected connection from "172.16.100.40:43996" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.317776 I | etcdmain: rejected connection from "172.16.100.28:44846" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.364844 I | etcdmain: rejected connection from "172.16.100.28:44850" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.380673 I | etcdmain: rejected connection from "172.16.100.40:44002" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.395944 I | etcdmain: rejected connection from "172.16.100.40:44004" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.428887 I | etcdmain: rejected connection from "172.16.100.28:44854" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.469235 I | etcdmain: rejected connection from "172.16.100.28:44858" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.485232 I | etcdmain: rejected connection from "172.16.100.40:44010" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.502181 I | etcdmain: rejected connection from "172.16.100.40:44012" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.535342 I | etcdmain: rejected connection from "172.16.100.28:44862" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.573931 I | etcdmain: rejected connection from "172.16.100.28:44866" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.589598 I | etcdmain: rejected connection from "172.16.100.40:44018" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.606645 I | etcdmain: rejected connection from "172.16.100.40:44020" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.639958 I | etcdmain: rejected connection from "172.16.100.28:44870" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.680285 I | etcdmain: rejected connection from "172.16.100.28:44874" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.694005 I | etcdmain: rejected connection from "172.16.100.40:44026" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.711096 I | etcdmain: rejected connection from "172.16.100.40:44028" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.720884 W | rafthttp: health check for peer d9c6974366ce1ba could not connect: x509: certificate has expired or is not yet valid
2019-07-18 11:25:02.724850 W | rafthttp: health check for peer 9019c714a7a7fe3b could not connect: x509: certificate has expired or is not yet valid
2019-07-18 11:25:02.748170 I | etcdmain: rejected connection from "172.16.100.28:44878" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.786614 I | etcdmain: rejected connection from "172.16.100.28:44882" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.800473 I | etcdmain: rejected connection from "172.16.100.40:44034" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.817778 I | etcdmain: rejected connection from "172.16.100.40:44036" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.852748 I | etcdmain: rejected connection from "172.16.100.28:44886" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.891056 I | etcdmain: rejected connection from "172.16.100.28:44890" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.904937 I | etcdmain: rejected connection from "172.16.100.40:44042" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.925306 I | etcdmain: rejected connection from "172.16.100.40:44044" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.960414 I | etcdmain: rejected connection from "172.16.100.28:44894" (error "remote error: tls: bad certificate", ServerName "")
2019-07-18 11:25:02.998057 I | etcdmain: rejected connection from "172.16.100.28:44898" (error "remote error: tls: bad certificate", ServerName "")
Results:
I was expecting rke to be succesfully deployed. However, there is tls: bad certificate error and health check problem.
sarpkoksal
All 12 comments
Is this a fresh installation or upgrade, in case of fresh installation make sure there are no old certificates deployed on the node, because rke will try to generate a new certificates and will use it to validate the cluster which will fail, can you also post the full logs of rke
galal-hussein
on 19 Jul 2019
This is a fresh installation. There is no old certificates deployed on the node, all nodes have been created freshly. I will post the full logs after I try again the "rke up" command because I currently lost logs.
sarpkoksal
on 22 Jul 2019
This is the full log after I run "./rke -d up" command:
root@hvl-virtual-machine:/home/hvl# ./rke -d up
DEBU[0000] RKE version v0.2.1
DEBU[0000] Parsing cluster file [# An example of an HA Kubernetes cluster for ONAP
nodes:
- address: 10.154.12.89
port: "22"
internal_address: 172.16.100.40
role:
- controlplane
- etcd
hostname_override: "onap-control-1"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.87
port: "22"
internal_address: 172.16.100.30
role:
- worker
hostname_override: "onap-k8s-1"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.73
port: "22"
internal_address: 172.16.100.22
role:
- worker
hostname_override: "onap-k8s-2"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.81
port: "22"
internal_address: 172.16.100.16
role:
- worker
hostname_override: "onap-k8s-3"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.80
port: "22"
internal_address: 172.16.100.14
role:
- worker
hostname_override: "onap-k8s-4"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.58
port: "22"
internal_address: 172.16.100.36
role:
- worker
hostname_override: "onap-k8s-5"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.59
port: "22"
internal_address: 172.16.100.38
role:
- worker
hostname_override: "onap-k8s-6"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.68
port: "22"
internal_address: 172.16.100.23
role:
- worker
hostname_override: "onap-k8s-7"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.102
port: "22"
internal_address: 172.16.100.18
role:
- worker
hostname_override: "onap-k8s-8"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.77
port: "22"
internal_address: 172.16.100.17
role:
- worker
hostname_override: "onap-k8s-9"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.65
port: "22"
internal_address: 172.16.100.45
role:
- worker
hostname_override: "onap-k8s-10"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.69
port: "22"
internal_address: 172.16.100.19
role:
- worker
hostname_override: "onap-k8s-11"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
- address: 10.154.12.103
port: "22"
internal_address: 172.16.100.47
role:
- worker
hostname_override: "onap-k8s-12"
user: ubuntu
ssh_key_path: "/home/hvl/onapkeypair.pem"
services:
kube-api:
service_cluster_ip_range: 10.43.0.0/16
pod_security_policy: false
always_pull_images: false
kube-controller:
cluster_cidr: 10.42.0.0/16
service_cluster_ip_range: 10.43.0.0/16
kubelet:
cluster_domain: cluster.local
cluster_dns_server: 10.43.0.10
fail_swap_on: false
network:
plugin: canal
authentication:
strategy: x509
ssh_key_path: "/root/onapkeypair.pem"
ssh_agent_auth: false
authorization:
mode: rbac
ignore_docker_version: false
kubernetes_version: "v1.13.4-rancher1-1"
private_registries:
- url: nexus3.onap.org:10001
user: docker
password: docker
is_default: true
cluster_name: "onap"
restore:
restore: false
snapshot_name: ""
]
INFO[0000] Initiating Kubernetes cluster
DEBU[0000] Host: 10.154.12.89 has role: controlplane
DEBU[0000] Host: 10.154.12.89 has role: etcd
DEBU[0000] Host: 10.154.12.87 has role: worker
DEBU[0000] Host: 10.154.12.73 has role: worker
DEBU[0000] Host: 10.154.12.81 has role: worker
DEBU[0000] Host: 10.154.12.80 has role: worker
DEBU[0000] Host: 10.154.12.58 has role: worker
DEBU[0000] Host: 10.154.12.59 has role: worker
DEBU[0000] Host: 10.154.12.68 has role: worker
DEBU[0000] Host: 10.154.12.102 has role: worker
DEBU[0000] Host: 10.154.12.77 has role: worker
DEBU[0000] Host: 10.154.12.65 has role: worker
DEBU[0000] Host: 10.154.12.69 has role: worker
DEBU[0000] Host: 10.154.12.103 has role: worker
DEBU[0000] [state] previous state found, this is not a legacy cluster
INFO[0000] [certificates] Generating admin certificates and kubeconfig
DEBU[0000] Writing state file: {
"desiredState": {
"rkeConfig": {
"nodes": [
{
"address": "10.154.12.89",
"port": "22",
"internalAddress": "172.16.100.40",
"role": [
"controlplane",
"etcd"
],
"hostnameOverride": "onap-control-1",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.87",
"port": "22",
"internalAddress": "172.16.100.30",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-1",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.73",
"port": "22",
"internalAddress": "172.16.100.22",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-2",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.81",
"port": "22",
"internalAddress": "172.16.100.16",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-3",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.80",
"port": "22",
"internalAddress": "172.16.100.14",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-4",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.58",
"port": "22",
"internalAddress": "172.16.100.36",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-5",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.59",
"port": "22",
"internalAddress": "172.16.100.38",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-6",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.68",
"port": "22",
"internalAddress": "172.16.100.23",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-7",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.102",
"port": "22",
"internalAddress": "172.16.100.18",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-8",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.77",
"port": "22",
"internalAddress": "172.16.100.17",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-9",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.65",
"port": "22",
"internalAddress": "172.16.100.45",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-10",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.69",
"port": "22",
"internalAddress": "172.16.100.19",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-11",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
},
{
"address": "10.154.12.103",
"port": "22",
"internalAddress": "172.16.100.47",
"role": [
"worker"
],
"hostnameOverride": "onap-k8s-12",
"user": "ubuntu",
"sshKeyPath": "/home/hvl/onapkeypair.pem"
}
],
"services": {
"etcd": {
"image": "nexus3.onap.org:10001/rancher/coreos-etcd:v3.2.24",
"extraArgs": {
"election-timeout": "5000",
"heartbeat-interval": "500"
},
"snapshot": true,
"retention": "72h",
"creation": "12h"
},
"kubeApi": {
"image": "nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1",
"serviceClusterIpRange": "10.43.0.0/16",
"serviceNodePortRange": "30000-32767"
},
"kubeController": {
"image": "nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1",
"clusterCidr": "10.42.0.0/16",
"serviceClusterIpRange": "10.43.0.0/16"
},
"scheduler": {
"image": "nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1"
},
"kubelet": {
"image": "nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1",
"clusterDomain": "cluster.local",
"infraContainerImage": "nexus3.onap.org:10001/rancher/pause-amd64:3.1",
"clusterDnsServer": "10.43.0.10"
},
"kubeproxy": {
"image": "nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1"
}
},
"network": {
"plugin": "canal",
"options": {
"canal_flannel_backend_type": "vxlan"
}
},
"authentication": {
"strategy": "x509"
},
"systemImages": {
"etcd": "nexus3.onap.org:10001/rancher/coreos-etcd:v3.2.24",
"alpine": "nexus3.onap.org:10001/rancher/rke-tools:v0.1.16",
"nginxProxy": "nexus3.onap.org:10001/rancher/rke-tools:v0.1.16",
"certDownloader": "nexus3.onap.org:10001/rancher/rke-tools:v0.1.16",
"kubernetesServicesSidecar": "nexus3.onap.org:10001/rancher/rke-tools:v0.1.16",
"kubedns": "nexus3.onap.org:10001/rancher/k8s-dns-kube-dns-amd64:1.15.0",
"dnsmasq": "nexus3.onap.org:10001/rancher/k8s-dns-dnsmasq-nanny-amd64:1.15.0",
"kubednsSidecar": "nexus3.onap.org:10001/rancher/k8s-dns-sidecar-amd64:1.15.0",
"kubednsAutoscaler": "nexus3.onap.org:10001/rancher/cluster-proportional-autoscaler-amd64:1.0.0",
"coredns": "nexus3.onap.org:10001/",
"corednsAutoscaler": "nexus3.onap.org:10001/",
"kubernetes": "nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1",
"flannel": "nexus3.onap.org:10001/rancher/coreos-flannel:v0.10.0",
"flannelCni": "nexus3.onap.org:10001/rancher/coreos-flannel-cni:v0.3.0",
"calicoNode": "nexus3.onap.org:10001/rancher/calico-node:v3.4.0",
"calicoCni": "nexus3.onap.org:10001/rancher/calico-cni:v3.4.0",
"calicoCtl": "nexus3.onap.org:10001/rancher/calico-ctl:v2.0.0",
"canalNode": "nexus3.onap.org:10001/rancher/calico-node:v3.4.0",
"canalCni": "nexus3.onap.org:10001/rancher/calico-cni:v3.4.0",
"canalFlannel": "nexus3.onap.org:10001/rancher/coreos-flannel:v0.10.0",
"weaveNode": "nexus3.onap.org:10001/weaveworks/weave-kube:2.5.0",
"weaveCni": "nexus3.onap.org:10001/weaveworks/weave-npc:2.5.0",
"podInfraContainer": "nexus3.onap.org:10001/rancher/pause-amd64:3.1",
"ingress": "nexus3.onap.org:10001/rancher/nginx-ingress-controller:0.21.0-rancher1",
"ingressBackend": "nexus3.onap.org:10001/rancher/nginx-ingress-controller-defaultbackend:1.4",
"metricsServer": "nexus3.onap.org:10001/rancher/metrics-server-amd64:v0.3.1"
},
"sshKeyPath": "/root/onapkeypair.pem",
"sshAgentAuth": false,
"authorization": {
"mode": "rbac"
},
"ignoreDockerVersion": false,
"kubernetesVersion": "v1.13.4-rancher1-1",
"privateRegistries": [
{
"url": "nexus3.onap.org:10001",
"user": "docker",
"password": "docker",
"isDefault": true
}
],
"ingress": {
"provider": "nginx"
},
"clusterName": "onap",
"cloudProvider": {},
"prefixPath": "/",
"addonJobTimeout": 30,
"bastionHost": {},
"monitoring": {
"provider": "metrics-server"
},
"restore": {},
"dns": {
"provider": "kube-dns"
}
},
"certificatesBundle": {
"kube-admin": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIC6TCCAdGgAwIBAgIIJZokmjciKjcwDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE\nAxMHa3ViZS1jYTAeFw0xOTA3MjIwOTMyNTRaFw0yOTA3MTkwOTQwNThaMC4xFzAV\nBgNVBAoTDnN5c3RlbTptYXN0ZXJzMRMwEQYDVQQDEwprdWJlLWFkbWluMIIBIjAN\nBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoTAmr6denrGVhtyN28NW26NGlKZM\n7TBRsFJbzXKsLws1QLq7p2aMijhEKm9WXgrryDuu5+KfjakCOzfCrvYtYQsF2leX\n6Sn5xtNxsVwCJ+hxrnfKn+wVGMfpdXYf7aN1wQmizeXzAzUOThRNeonsp6bwfkBk\nzYDtEMnRZ3edLppoNAVmO07KRh89Af+MwsAOm5glHyqCNlyVzkPQ3rRzSnOg59XX\nR0C6Wxx6LYIUuLTDEh7UEeQ9iyItex24QiQYVdWPHYXEVgUR/TG1ExlYGkaUjR+Y\n/i66IWf8SF7KUy5vBFFTqsltHukU612tG5ze4ALlmzhMbInXuHlSuPnQCQIDAQAB\noycwJTAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwIwDQYJKoZI\nhvcNAQELBQADggEBAA1g+T98kGygD7aQ77wQnrWFJySiPCmTRax+w5/dPQpdRoiK\nePG424WchhEfpV/gHyG3sb9NLSPde/sa1v3dlKXMXcoLqeQ26oOE/eWcXb/QUrNI\nMIAoAae0/N5OHfHxbTR6CO8omfJ7SZWUmt9sWXd9rorwtxn7KU8F3jon+HFOwB1Z\nm7zUH6/X892T0QZe6OXtstH20B3cY3GhoiWYKO+OoIZCu1pFUQoGf3Xki060tzxm\nhEspmOBHh4Ddj6oXNyhlYPHYjyaY0QxjDd68iTLvaGB2m8K6oDOGVo+yShOvQkl/\nG7GPHPYdrSrZ/w0RVMmF8JMHJ2uO9LJaLX3m9RY=\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAoTAmr6denrGVhtyN28NW26NGlKZM7TBRsFJbzXKsLws1QLq7\np2aMijhEKm9WXgrryDuu5+KfjakCOzfCrvYtYQsF2leX6Sn5xtNxsVwCJ+hxrnfK\nn+wVGMfpdXYf7aN1wQmizeXzAzUOThRNeonsp6bwfkBkzYDtEMnRZ3edLppoNAVm\nO07KRh89Af+MwsAOm5glHyqCNlyVzkPQ3rRzSnOg59XXR0C6Wxx6LYIUuLTDEh7U\nEeQ9iyItex24QiQYVdWPHYXEVgUR/TG1ExlYGkaUjR+Y/i66IWf8SF7KUy5vBFFT\nqsltHukU612tG5ze4ALlmzhMbInXuHlSuPnQCQIDAQABAoIBAGPRltOGeXEbFjcr\nFGp4wKxJo6bgiZkFKEFh71n+XqTPcw3/1b/oILZj+8qvojOg3bzlQP52+EnY1Pnp\nIRYKe7WyBtDdF8Ub6oN3Y9vq16bxKp1Zr9Bzeekbu/ayzTjz1jNSDVEmyuQ8kfxG\nVGjDSxvTpPD6XbJp2eVgpd5GU7+Mgbs2VQndzzBP2ijJlf7naCWr4Ry1HlqMTdXh\nqRHRl2rF7/hO5bf5Vv1HvM/P2Ycy94q4v2KqqvaGEod8XkiIYrG1wCrOcvWAuyn2\nCclWh8SB3BAre4bkp4B5W3U0lbzI1K6T6bMet5tR7+QrNh7pdScomCOetYcSQYyp\nR2mh2JECgYEAz/icZ6nO6OhzbgjgcaK6QRowQ/EZTshtIBxy9KAh47dyCNynw4WX\niledCkAd2MDAlaPYu88DVW8FvYNW+QNtIZ0BksLFCJpuaBtwtWZQVAy6S0dQXbrF\ncCbUkzk3UAJLPM8k8QZZoqfX1Q/dppr8VpwqmWJL2/drXdOZipscej0CgYEAxmmy\nLOKJQNh1H6bJvpsypoklLTDhBZyvLNBN7KQ8lBNffIPd8x1c0eKxY0BnVeSi6PmH\nNYBY44Kf2KxJLh3t2jQ8gyKNoSIfeXcrq7DyQvQcdxRSXxue8LEaoudtgB+BwEZJ\n8q+WDi9Tb3FCDmYjHwdpdmAte96eFZUZ42K35b0CgYEAlimUBOV/GHx+m/yc0nAE\nu+0E3InBFDlzUbQ47JiS37EdDXkFSQ+0k5cfNL+GzXKfm2xqmXCnn5BCPMwQiEwJ\nCDQtZqJrUgLvpQvrHY96gLWP7brYNFUfKhNfec3eOk7ttEJ9Eerb3a4DETwbdqiB\neqfXulhXHcRQKbcXLhVBQqECgYEAn/CB1VbDZ0dVmq8bP4/hk4dINZaKPbjhKNG9\njv2Ra5f2R+wQlTJdKT+A2M4Ivp9whzbmPEV64eLhMYsn7onnuxaSlO7clkBoeMSM\nqnkXwMsAyyjRgaNFSyaonxrJFRfQwwql/4B1CHgz3MjCg9LnitKxu9+pqol4i1NG\nrBpevU0CgYAqluqe0pvSwHZGE55oMmv3eiOTk5NQkfuIiBIpE0/4/1nI67VZB4Qi\nZ44ZPn77mz/cPVrz++0QnoZNBz1KXlUf2lROerCam57Je3IX9RkStjnbZOuuIvx6\n8KfQBCq3VHcUpTI7klEhZq4MUW2A8l0FJ+231SMWbPP7/3ceEFU8bg==\n-----END RSA PRIVATE KEY-----\n",
"config": "apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n api-version: v1\n certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN3akNDQWFxZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFTTVJBd0RnWURWUVFERXdkcmRXSmwKTFdOaE1CNFhEVEU1TURjeU1qQTVNekkxTkZvWERUSTVNRGN4T1RBNU16STFORm93RWpFUU1BNEdBMVVFQXhNSAphM1ZpWlMxallUQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUxzR2czdk1BMFJNCm5wdjFFamY0UFg3WXBBbVIwREdlanhTNGNZa3BZYWYzRVBwSFhiSWJPRm1WUzVldGcxYm4zeWRhV1U2ZzQrS1EKMDllQ1hEakVUbjBKVjFtSlZ1TFhoaG0rUWtqOWVpRGI2UU1pcWwxbE81OWFKV0xPTHdrdlJDblpISTNuY0lEVwpVY0t0OVJsN0F3ZDNUQWZSSFFTTHVFdWVkSmlEYnR4alFRUzNpdFZOSjNLaFZkZWFoTjl6Qy9NUWlxUmQ4OGpsCjZFcEZjSnp3eERHR1VFc0N3cUV0V3NGVXgrNS9iam1xRXBUbUlIbWlVWkFteUJhYjFSMERua3JDYVdXeFpHWmcKUm9kQlhyU0RpVnlBOW1MWEJiWDdNdSt0TURIa1hSY3FmclRNN0FMWkIvYThWNWJQQkcrNEFtb0pRNE9lRStsTQpVbTFIRVRuOUVrVUNBd0VBQWFNak1DRXdEZ1lEVlIwUEFRSC9CQVFEQWdLa01BOEdBMVVkRXdFQi93UUZNQU1CCkFmOHdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBRjMvWWdnRlFaR00weTdrTU5Cb1l1ODBCWmtyTjMyUU9TbzcKQVNURzFYZ2FwQ0o2OG1tbzVVd1NZbERkQ09jRUErWXBBOHBQVExOd1NuR1ZCRDZPcVVoeWZHMXU0THJ6N09qVApvcnpFd05HZXhqUlFCQ2ZWNDVtdnpIeEsrQkxwbHBMeVpKMjd2ZWZGN0FFSHFMK1RnNDNqWlVkWDZuK3JFb1hvCm0xTUVzYXpqd0VGS0VkZStvcnFtM1d5MFgwUHNkSGo0U0Z0SUVNSHl4WlR4NytpMjJoSE1IOUxDQVpVTStGZEEKdjl0MnNjUWxqb1loQWRseHJSbkt0dDZDK2hldGtTN2JQVlIwRCsxL3NxMUwzTFdZVTB3S2tDWitVdUwzeXJhcAoxSlAyRVMxb3kyYlVxQi8xcjlRMDNmU1JuOStzblhoMmkzdmtza0kybmlIRThpdnBEYTA9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K\n server: \"https://10.154.12.89:6443\"\n name: \"onap\"\ncontexts:\n- context:\n cluster: \"onap\"\n user: \"kube-admin-onap\"\n name: \"onap\"\ncurrent-context: \"onap\"\nusers:\n- name: \"kube-admin-onap\"\n user:\n client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM2VENDQWRHZ0F3SUJBZ0lJSlpva21qY2lLamN3RFFZSktvWklodmNOQVFFTEJRQXdFakVRTUE0R0ExVUUKQXhNSGEzVmlaUzFqWVRBZUZ3MHhPVEEzTWpJd09UTXlOVFJhRncweU9UQTNNVGt3T1RRd05UaGFNQzR4RnpBVgpCZ05WQkFvVERuTjVjM1JsYlRwdFlYTjBaWEp6TVJNd0VRWURWUVFERXdwcmRXSmxMV0ZrYldsdU1JSUJJakFOCkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQW9UQW1yNmRlbnJHVmh0eU4yOE5XMjZOR2xLWk0KN1RCUnNGSmJ6WEtzTHdzMVFMcTdwMmFNaWpoRUttOVdYZ3JyeUR1dTUrS2ZqYWtDT3pmQ3J2WXRZUXNGMmxlWAo2U241eHROeHNWd0NKK2h4cm5mS24rd1ZHTWZwZFhZZjdhTjF3UW1pemVYekF6VU9UaFJOZW9uc3A2Yndma0JrCnpZRHRFTW5SWjNlZExwcG9OQVZtTzA3S1JoODlBZitNd3NBT201Z2xIeXFDTmx5VnprUFEzclJ6U25PZzU5WFgKUjBDNld4eDZMWUlVdUxUREVoN1VFZVE5aXlJdGV4MjRRaVFZVmRXUEhZWEVWZ1VSL1RHMUV4bFlHa2FValIrWQovaTY2SVdmOFNGN0tVeTV2QkZGVHFzbHRIdWtVNjEydEc1emU0QUxsbXpoTWJJblh1SGxTdVBuUUNRSURBUUFCCm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0l3RFFZSktvWkkKaHZjTkFRRUxCUUFEZ2dFQkFBMWcrVDk4a0d5Z0Q3YVE3N3dRbnJXRkp5U2lQQ21UUmF4K3c1L2RQUXBkUm9pSwplUEc0MjRXY2hoRWZwVi9nSHlHM3NiOU5MU1BkZS9zYTF2M2RsS1hNWGNvTHFlUTI2b09FL2VXY1hiL1FVck5JCk1JQW9BYWUwL041T0hmSHhiVFI2Q084b21mSjdTWldVbXQ5c1dYZDlyb3J3dHhuN0tVOEYzam9uK0hGT3dCMVoKbTd6VUg2L1g4OTJUMFFaZTZPWHRzdEgyMEIzY1kzR2hvaVdZS08rT29JWkN1MXBGVVFvR2YzWGtpMDYwdHp4bQpoRXNwbU9CSGg0RGRqNm9YTnlobFlQSFlqeWFZMFF4akRkNjhpVEx2YUdCMm04SzZvRE9HVm8reVNoT3ZRa2wvCkc3R1BIUFlkclNyWi93MFJWTW1GOEpNSEoydU85TEphTFgzbTlSWT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=\n client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBb1RBbXI2ZGVuckdWaHR5TjI4TlcyNk5HbEtaTTdUQlJzRkpielhLc0x3czFRTHE3CnAyYU1pamhFS205V1hncnJ5RHV1NStLZmpha0NPemZDcnZZdFlRc0YybGVYNlNuNXh0TnhzVndDSitoeHJuZksKbit3VkdNZnBkWFlmN2FOMXdRbWl6ZVh6QXpVT1RoUk5lb25zcDZid2ZrQmt6WUR0RU1uUlozZWRMcHBvTkFWbQpPMDdLUmg4OUFmK013c0FPbTVnbEh5cUNObHlWemtQUTNyUnpTbk9nNTlYWFIwQzZXeHg2TFlJVXVMVERFaDdVCkVlUTlpeUl0ZXgyNFFpUVlWZFdQSFlYRVZnVVIvVEcxRXhsWUdrYVVqUitZL2k2NklXZjhTRjdLVXk1dkJGRlQKcXNsdEh1a1U2MTJ0RzV6ZTRBTGxtemhNYkluWHVIbFN1UG5RQ1FJREFRQUJBb0lCQUdQUmx0T0dlWEViRmpjcgpGR3A0d0t4Sm82YmdpWmtGS0VGaDcxbitYcVRQY3czLzFiL29JTFpqKzhxdm9qT2czYnpsUVA1MitFblkxUG5wCklSWUtlN1d5QnREZEY4VWI2b04zWTl2cTE2YnhLcDFacjlCemVla2J1L2F5elRqejFqTlNEVkVteXVROGtmeEcKVkdqRFN4dlRwUEQ2WGJKcDJlVmdwZDVHVTcrTWdiczJWUW5kenpCUDJpakpsZjduYUNXcjRSeTFIbHFNVGRYaApxUkhSbDJyRjcvaE81YmY1VnYxSHZNL1AyWWN5OTRxNHYyS3FxdmFHRW9kOFhraUlZckcxd0NyT2N2V0F1eW4yCkNjbFdoOFNCM0JBcmU0YmtwNEI1VzNVMGxiekkxSzZUNmJNZXQ1dFI3K1FyTmg3cGRTY29tQ09ldFljU1FZeXAKUjJtaDJKRUNnWUVBei9pY1o2bk82T2h6YmdqZ2NhSzZRUm93US9FWlRzaHRJQnh5OUtBaDQ3ZHlDTnludzRXWAppbGVkQ2tBZDJNREFsYVBZdTg4RFZXOEZ2WU5XK1FOdElaMEJrc0xGQ0pwdWFCdHd0V1pRVkF5NlMwZFFYYnJGCmNDYlVremszVUFKTFBNOGs4UVpab3FmWDFRL2RwcHI4VnB3cW1XSkwyL2RyWGRPWmlwc2NlajBDZ1lFQXhtbXkKTE9LSlFOaDFINmJKdnBzeXBva2xMVERoQlp5dkxOQk43S1E4bEJOZmZJUGQ4eDFjMGVLeFkwQm5WZVNpNlBtSApOWUJZNDRLZjJLeEpMaDN0MmpROGd5S05vU0lmZVhjcnE3RHlRdlFjZHhSU1h4dWU4TEVhb3VkdGdCK0J3RVpKCjhxK1dEaTlUYjNGQ0RtWWpId2RwZG1BdGU5NmVGWlVaNDJLMzViMENnWUVBbGltVUJPVi9HSHgrbS95YzBuQUUKdSswRTNJbkJGRGx6VWJRNDdKaVMzN0VkRFhrRlNRKzBrNWNmTkwrR3pYS2ZtMnhxbVhDbm41QkNQTXdRaUV3SgpDRFF0WnFKclVnTHZwUXZySFk5NmdMV1A3YnJZTkZVZktoTmZlYzNlT2s3dHRFSjlFZXJiM2E0REVUd2JkcWlCCmVxZlh1bGhYSGNSUUtiY1hMaFZCUXFFQ2dZRUFuL0NCMVZiRFowZFZtcThiUDQvaGs0ZElOWmFLUGJqaEtORzkKanYyUmE1ZjJSK3dRbFRKZEtUK0EyTTRJdnA5d2h6Ym1QRVY2NGVMaE1Zc243b25udXhhU2xPN2Nsa0JvZU1TTQpxbmtYd01zQXl5alJnYU5GU3lhb254ckpGUmZRd3dxbC80QjFDSGd6M01qQ2c5TG5pdEt4dTkrcHFvbDRpMU5HCnJCcGV2VTBDZ1lBcWx1cWUwcHZTd0haR0U1NW9NbXYzZWlPVGs1TlFrZnVJaUJJcEUwLzQvMW5JNjdWWkI0UWkKWjQ0WlBuNzdtei9jUFZyeisrMFFub1pOQnoxS1hsVWYybFJPZXJDYW01N0plM0lYOVJrU3RqbmJaT3V1SXZ4Ngo4S2ZRQkNxM1ZIY1VwVEk3a2xFaFpxNE1VVzJBOGwwRkorMjMxU01XYlBQNy8zY2VFRlU4Ymc9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=",
"name": "kube-admin",
"commonName": "kube-admin",
"ouName": "system:masters",
"envName": "KUBE_ADMIN",
"path": "/etc/kubernetes/ssl/kube-admin.pem",
"keyEnvName": "KUBE_ADMIN_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-admin-key.pem",
"configEnvName": "KUBECFG_KUBE_ADMIN",
"configPath": "./kube_config_cluster.yml"
},
"kube-apiserver": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIDfzCCAmegAwIBAgIIA4hnAuhDgCAwDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE\nAxMHa3ViZS1jYTAeFw0xOTA3MjIwOTMyNTRaFw0yOTA3MTkwOTMyNThaMBkxFzAV\nBgNVBAMTDmt1YmUtYXBpc2VydmVyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB\nCgKCAQEAk+mykVxcCjphPwC6CC3LL7ZD1Gqa8vEUVulXc3VBJV7UGKpL+Pgh7oaF\nXk4OMSRfy9VDjLdZ/ZoMGVkzEE2JRQ7oUQ2abT0jnGHynVmo6c49otKnysCUoxdt\ntTKmlDXh0MAV0AKti0biKsRplWnI51Su3eXYqPX8Ry4R8v1hwPa3ltc+NWcl+7fq\nFDLEoOTrmmfEe1UumyAoVUAoJT8WHtsiQ59I/TktWJDzUzOazkWVUpyiEbE0uKV7\n3v8LRhfTwnMYI6RQQbzq7SBopD9usKO/RmR0TY9RXjZMUzajfxM431kiIclTNDd/\nNIGD4cklocvQYuWQiNm2JwUeqZixRwIDAQABo4HRMIHOMA4GA1UdDwEB/wQEAwIF\noDAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwEwgZwGA1UdEQSBlDCBkYIO\nb25hcC1jb250cm9sLTGCCWxvY2FsaG9zdIIKa3ViZXJuZXRlc4ISa3ViZXJuZXRl\ncy5kZWZhdWx0ghZrdWJlcm5ldGVzLmRlZmF1bHQuc3ZjgiRrdWJlcm5ldGVzLmRl\nZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWyHBAqaDFmHBKwQZCiHBH8AAAGHBAorAAEw\nDQYJKoZIhvcNAQELBQADggEBAJZzXRdfSge5etpph8mbzFbVemoXhXloZp4WgXk2\nNiwkY8M7YE9kln4iuO2ETEKA9zgatSsNNys5PynmjwmANnGgiAyBCi1TLvr/PIOK\nMlUa0u9MLKPLdO6KeDM/YKCVnXRBbHeH27w2CHw59llJgrV8uou8i7C2m7siuWVH\ntrYSjgLt65/Z4CpczkaZo86M1t4msECGo+bda4xrl4QtfILzZRQiAHaqhJyVbByB\nAby/z67Iv1yZ8thbmLQpNk1wSXdVKdR6TQLX2agks0meJ8ynLNofqmPlHjm5Y9z7\nVSlbSR9IUI40fzZqtGHuQ7do+QTh33LEbyvee5W49LEr6GI=\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAk+mykVxcCjphPwC6CC3LL7ZD1Gqa8vEUVulXc3VBJV7UGKpL\n+Pgh7oaFXk4OMSRfy9VDjLdZ/ZoMGVkzEE2JRQ7oUQ2abT0jnGHynVmo6c49otKn\nysCUoxdttTKmlDXh0MAV0AKti0biKsRplWnI51Su3eXYqPX8Ry4R8v1hwPa3ltc+\nNWcl+7fqFDLEoOTrmmfEe1UumyAoVUAoJT8WHtsiQ59I/TktWJDzUzOazkWVUpyi\nEbE0uKV73v8LRhfTwnMYI6RQQbzq7SBopD9usKO/RmR0TY9RXjZMUzajfxM431ki\nIclTNDd/NIGD4cklocvQYuWQiNm2JwUeqZixRwIDAQABAoIBAHNTxPpX1B/Zcxdn\nnqVf50w1Rd/qmxQPard1Ss1oQDo+HQt6RLBD8FkFpBlqrOAeJ19L0lZ2Y4E7ynE9\nPnZYrk4dGICSJbVE9W4qZLvRO8XJG//HVjLdQ0eCrfSZII7xEDfufVNpsxykURXd\nfSGVkKLmQU4ct3jOCz0Iq9AVRlHoHLxk40vfsr7N1GGdrlJTiJ1tmQOfIxN34EDx\nDhAeX0an2YjDxfaRCiMVbeykfMGeJ6snNwB9l8j72+QdDsKn9Nef5SX9JOiptGb4\nHVWbrumbKhZIVsYadIGRldKhIuPSI5YYbKj24j/S0QFBN6/HZ/u2ZI4nqSgk/KXt\nraw9DKECgYEAwn6sBMSm2HrqiKjwiIBGMaSouL/mpKSjIDuLzhF/3qhng04NCGH+\n4OLjHyqgPQ9nvWhP3ke2ifxX5Z9Z6nVdTYsN+emzwHVisfW9bjTYdnskMRc8xdCH\nucNiy/MwvjWQ9ER2570PLOKLuGxTA2/bPop1ad6nvgWptZLkOnIdIdECgYEAwq/8\nNpNzHMT9KKt4V4gMvmTVl9UaB1onvqqdItPImSKe4qULwN9oPuwf9Xa+J1KbxEKK\nHXPL0roAFGM4cH0rkd3QiT42Lz+ujv4gmc5Rwpd9vk1IRUYM0IB9H8YBIO9dqzOD\nxeL+Vo+HFwjwMvosL88bruvx6oAw3XCSW3emj5cCgYEAqehHFhNFPjCC2sfa+SQ2\nLgAOdPcP4h0BdJMQWwDml/i+FPMGZ6BULNmIWB3AxhbDJbAOtbwoNvr/PlWrEytc\nSvWDTCNhKGxjv9tRlQkNYNPKligaqyS0rQ+HK2xT1Jau/vUgeumqbCesr3vSOVpq\nSYRjdg1QtKv1ye18+6Hc2UECgYBjXwZCKRLJbXa2SodbTjMK/1bSb9AgTpEYzeb0\nvtqV50jn8ExGfwGrlERfuuMmGREupLPwgolXnt6piVdme01A39IwKxPnG/8SN34W\nY2vyAfcJqlKoITArQFodXKh40hSibOHmoI0nS40tYb+QlYb2RCUiLIZHJnVKocni\na/kVMwKBgGn7GwHV9ixAYKnIoSAcACwo8+jxlzj9RS/nc7Druro2MO3qdnWrZbhk\nsZTKTGm3YA9voWN2oD3b0R4p+3kllvnzTfr7eNpwXWkNCY39SoCwyL1cW7lcbruM\nUv647MpyrYcZ4tH9G9dQOGetGiSQZiTkfvdZazoDrcNGhtdaLMge\n-----END RSA PRIVATE KEY-----\n",
"config": "",
"name": "kube-apiserver",
"commonName": "system:kube-apiserver",
"ouName": "",
"envName": "KUBE_APISERVER",
"path": "/etc/kubernetes/ssl/kube-apiserver.pem",
"keyEnvName": "KUBE_APISERVER_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-apiserver-key.pem",
"configEnvName": "",
"configPath": ""
},
"kube-apiserver-proxy-client": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIDAzCCAeugAwIBAgIIRtb/uVzUGKgwDQYJKoZIhvcNAQELBQAwKjEoMCYGA1UE\nAxMfa3ViZS1hcGlzZXJ2ZXItcmVxdWVzdGhlYWRlci1jYTAeFw0xOTA3MjIwOTMy\nNTRaFw0yOTA3MTkwOTMyNTZaMCYxJDAiBgNVBAMTG2t1YmUtYXBpc2VydmVyLXBy\nb3h5LWNsaWVudDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKWr+sl1\n5q3+vbJ/OjjKvJcmZF8r012WryMT47UcG5sdoo2rlnv9Sa5kNkexYkip/yl5AzGT\nLWhSdJPUSO0y4DtHPSY97QzW9D9GKCFYYrZiCDHoILWXQS3FGBvDLeMLeVa1Ftp1\nZII30IJ5OD6klPtIBq/KV+lgzsMeMVqSEy2PJfoWVLkUsvac4LOkf2p86YnmoY4J\nxD3ZjgKVdOXpTkeTH9b6VxvIaxGiNdTtbHyxB1Gv7jk+lYbWrKxCrj2EURmFQPFA\nDQ4iZSDFgrBWOk/UzVxPqvQAEreHu7e0R7n4/Q0nsP5gZSHlbMEgaZP/ZhCn1f/x\ndoQwhfgGxfpcEe8CAwEAAaMxMC8wDgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQG\nCCsGAQUFBwMCBggrBgEFBQcDATANBgkqhkiG9w0BAQsFAAOCAQEAye1I4LjQ1pc1\nSD+JUlfAjXQu7EE0t4ORT4AQBi9RBwstsdw5fHOl1eik3RZO6WbX2zXBcnZqn4TW\nE4wD1hxoohdRIgeq7iqDAGT5gsgNCMlnpcEUqAGdkEJ1F2hrIELrG+d5xuwx10Lm\nbZARAmLzXBsQ8Owk8zrT40d4zzgcbN4fzUhTwYMA3nguwZdORT6Qd6B8WY/omqhc\nlZ1g+PJ3gvlQTmTFOzKROne154m1SLbSzQbNO9lWRsy0NtqBM7xX2zsN364I8E2i\nud+aj7AcWbRqwS/Q+TIPDAtSN8/1+myFSH/8s5i9gVILU1fNPd7U+dqrFgErocIC\nyNaGENFKdQ==\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEogIBAAKCAQEApav6yXXmrf69sn86OMq8lyZkXyvTXZavIxPjtRwbmx2ijauW\ne/1JrmQ2R7FiSKn/KXkDMZMtaFJ0k9RI7TLgO0c9Jj3tDNb0P0YoIVhitmIIMegg\ntZdBLcUYG8Mt4wt5VrUW2nVkgjfQgnk4PqSU+0gGr8pX6WDOwx4xWpITLY8l+hZU\nuRSy9pzgs6R/anzpieahjgnEPdmOApV05elOR5Mf1vpXG8hrEaI11O1sfLEHUa/u\nOT6VhtasrEKuPYRRGYVA8UANDiJlIMWCsFY6T9TNXE+q9AASt4e7t7RHufj9DSew\n/mBlIeVswSBpk/9mEKfV//F2hDCF+AbF+lwR7wIDAQABAoIBAClbTTeNXxCATwqI\nCBi1Q27X/qZjYlgyjXXMOUO0yI9oTnPfKam6CPKnwYUUGtzY0wGfW9S8UNGCA+gD\nTlEoKiGiHvm068AQyNgiw1HPppSw1CrhL4klQPgUA3iquIG9E4U5n4LZW0XGeR6Y\n9BlvLGGuET1A3cE8kWDT4K4jWPY+JwD+HTiGVjSiRVH24mHTmZ8Kd7C1c0CyFId6\n0pQ7kl1iy8zfl9IlQn7IO6ZVhN0ooH7wIg7dsuTkxrzh80Hg+kJAemRPFrJ6aarH\nTwpqC4pw6wtF1c2EZGhITwxE9xB/4poYqF6UYVFW9n495l0+VxEFI1fEY/5h1Wdi\nqZzcAUECgYEAy3xb3+H0vZ1I5EHgYKDNB+S4lM68w6k97Wc+5pEtzulNoSD+QoVO\n+ht/W6P5PDnA//bShDLo4JWnDKAJwAb7blXajCW3s1Nn1L2mzqmhNvm/657+BzSd\nGnqg3GK+TiF48/+kKmYxG+cOclBBZJYSdx9YGWBFAQmWc6AtQP85/qUCgYEA0G1f\n2gKB/zEjAkMBxIT+MTh07GZ3IIgBke+Yvuq7JNh+r2mR2fmKRLVfp0OWgFZBTzeN\nqCGg6hrAh9yESembGQsajCkliKYOXzvsGcX4ZMiuBTnc06Pv5WMiWFAEEYufaGlY\n7MGLzFUftmKIEfY3OY/aXykBBiXC8ocK84Z/3gMCgYAz2HmoL3AmJ778uXQr6oyS\n6EnHUG10BNkMdfPm1zKut27aEQf3dvPKcdsfREYqfCSakw7MNBR1tGudoeEaFi1V\nzXYjozCM9kFMSQ4JNdsnoLavmBDtURB+QKRfwqDuU0T4FjiDvQniLBrYMW6X7EzE\nIS5XONocC49T4onlN6JVKQKBgCEMtt5C94oXqI17QpR+4i+73A6AUPN/nYAelgHw\nBqzZB32ZJ9+AikRK3lgPh5XiF4kQLg4zBukYRN9GQFPYDxfxgswm2iTI8A552f5c\nPf79M1rwyetlW2zsQ03tojeGBEjyK3+fUSOde+HXzevLNwaRoPbl8Zuf8UGz2DC1\nayExAoGAOHTsT5s8CyYbUH2c42ywKAON4Ll2XUhaaXGy245e+zZ/tnfCY0rCy+2/\nhxYj91OSuLEBs07WcHyXHe4G/Vp1S7IWB0SLrtXLlxRmIA+pBsqZmDmuux2/ay04\nP8UoHsC7WOG8qvEL3dAZu2OL9AODjWDFqiDVF+N0fWFXaqvHaHk=\n-----END RSA PRIVATE KEY-----\n",
"config": "apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n api-version: v1\n certificate-authority: /etc/kubernetes/ssl/kube-ca.pem\n server: \"https://127.0.0.1:6443\"\n name: \"local\"\ncontexts:\n- context:\n cluster: \"local\"\n user: \"kube-apiserver-proxy-client-local\"\n name: \"local\"\ncurrent-context: \"local\"\nusers:\n- name: \"kube-apiserver-proxy-client-local\"\n user:\n client-certificate: /etc/kubernetes/ssl/kube-apiserver-proxy-client.pem\n client-key: /etc/kubernetes/ssl/kube-apiserver-proxy-client-key.pem",
"name": "kube-apiserver-proxy-client",
"commonName": "system:kube-apiserver-proxy-client",
"ouName": "",
"envName": "KUBE_APISERVER_PROXY_CLIENT",
"path": "/etc/kubernetes/ssl/kube-apiserver-proxy-client.pem",
"keyEnvName": "KUBE_APISERVER_PROXY_CLIENT_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-apiserver-proxy-client-key.pem",
"configEnvName": "KUBECFG_KUBE_APISERVER_PROXY_CLIENT",
"configPath": "/etc/kubernetes/ssl/kubecfg-kube-apiserver-proxy-client.yaml"
},
"kube-apiserver-requestheader-ca": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIC8jCCAdqgAwIBAgIBADANBgkqhkiG9w0BAQsFADAqMSgwJgYDVQQDEx9rdWJl\nLWFwaXNlcnZlci1yZXF1ZXN0aGVhZGVyLWNhMB4XDTE5MDcyMjA5MzI1NFoXDTI5\nMDcxOTA5MzI1NFowKjEoMCYGA1UEAxMfa3ViZS1hcGlzZXJ2ZXItcmVxdWVzdGhl\nYWRlci1jYTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANi7PtInHz3A\neBV2I/gk6FNhECM2PB3KQ21JofUNxdL+xGJw5IdY8vbUIMmcyUuZ9UpXWPkCyu6u\n8SiYDOIxtiEaFaYYmwO3+8Sb3gbzJltK/Pvtoc0MBTzn6gqmtQsrOXR0+AUa+wV7\nRKIU7f4jOiXMdmqyNHgi6Ddax+XniAsDPDwNvsaH+T9+OXNXYv7GmZ1T98HqZYyc\nSrpz9oZa59bTQXS9BZFWcacVg0m0u/xOIhzLfxEG81pq9LjhF7EM+WcwXaT4ySOX\nFGmGEpMaXh/UcQ72VzK523OGCOOIlsfkSxWwCBZlPLENOSogRS/DIzGA9m4ry422\nHyhcPA6DZ38CAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB/wQFMAMB\nAf8wDQYJKoZIhvcNAQELBQADggEBAM+8UVjWSOM6JQo8Ct0a6hSN11YKYC1hp1jW\nGAHJEeEh4jiuRfItm889vwdVa8uqfVNygLj81jVSBcnbLFSET6rvRvwr4zRI2+2D\nlxemP7pxt8bq1X8rnZNR31o1/TrA8CFVsabwfkjMy6+foHqUht6RGjgVdyOPP7IE\nvfBUyp4eDibMhSJGhRQCIhlWniycM1DJalqbuUl6KJ6zr4+suv6BpAymA5wVD3Pd\n6GUIZ9PeejYOD6CM1dhvZWwIMtVFpOxCXE66KUUb4MOXtRim8cMNICw6npuTvIm/\np1Jf1ssrodkIWvOL5YAOQQ3Jw2e5963EeBSfssF0ouZd7MZXLJw=\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEA2Ls+0icfPcB4FXYj+CToU2EQIzY8HcpDbUmh9Q3F0v7EYnDk\nh1jy9tQgyZzJS5n1SldY+QLK7q7xKJgM4jG2IRoVphibA7f7xJveBvMmW0r8++2h\nzQwFPOfqCqa1Cys5dHT4BRr7BXtEohTt/iM6Jcx2arI0eCLoN1rH5eeICwM8PA2+\nxof5P345c1di/saZnVP3wepljJxKunP2hlrn1tNBdL0FkVZxpxWDSbS7/E4iHMt/\nEQbzWmr0uOEXsQz5ZzBdpPjJI5cUaYYSkxpeH9RxDvZXMrnbc4YI44iWx+RLFbAI\nFmU8sQ05KiBFL8MjMYD2bivLjbYfKFw8DoNnfwIDAQABAoIBAEEHbcIAmw8cQOuv\nlvtNl6fs57qo+0C0/6Y7Aa1VhJXae3JODUPPHJeS7oOP15Yni8HSk4+jluAm/ZUA\nil0ULWGhQf2kRF7JRXGokuXi7w9VYLL2iiXMBYZQz5iCjK/dOFAJCKH/hnIOmnlH\nbmFikH/m6WD6N2QUlTiGtJP29IrY0BdO409sfezJ+j5tDF9nmq0I+CqUHmlG+9XY\n4BjLe711JJbzSeIr3bKQsQYymKNe2sqzqE02/nr/UwBXAj8YxhvwHL7shIpZQ5Rd\nQDT2udKB3oKfTG6Ge7w8He9ftCgbCPy2Vwma9nqrCLO2Qp0btHaLwYeDCQTzZzZw\nFu+jXYECgYEA+RRps+jSdENuVA3pn3uDSwonQOUda7u+7naX+43ddyBjOvneVl8z\nXOd81BSseHqSNMvCqi5kFkAu/cGTX1vky0erVmn7J0BwGq69iv41GuJsfJ4LgeGR\ntOaEsU5oCvXb4Tmn+392Hhwj8HvPO04N++lgdc7Ln0FMeWxJaqeNeM0CgYEA3sDB\nEO3ALWDGsQDu+iIkw5QrMEE/1xBTZk4BMEYI8R3zb4OEEDE6nt8SE4SONyOxiTVZ\n0iqCN3p/RX/RVOTyyalVNvGMew3h2NlkfINO4uKkw9lNya46jj+fucARS1qwFJBt\ncTLsnmd7d1s39ddQhjyq6pq70WA/aZ3GaAGX0XsCgYEA4RQ3KKJdez5r9nzT/pAI\nZsj4C8Z+vZncwXRdhq390OZthR/CcN56dZuhy6wa5owsjpUKheqB+zFoxVolqGhL\n6f4LsHz4vdGW/CWnlnB84Jx3WG0hDO8fUg91HRwzHKRrW61URQo7XJ4lQ+09qOOx\n81BDIHIMww6o4pieB0j9qIkCgYBYpzhvp1/7729ElQ2Hib0+BaNmsiFZU8sNhXAd\nqVH3vKbdM+eElkMxZbCJChGbyl3nMM+NwJNQxzUOEP/cHL6wGe+rTVlOxEiliGfg\n0lW8IN3YgIzf/rKZWKMWR2Xaqr43gP+9tLnNlXvjFe/grRAETPLtKiYj/31daIjF\nbhThLwKBgC6fD/P8gN/6DjHbpF5OfvX7hz8hJ6L6recuM0UXUaeTc3YnmdND3BfI\nSUFtdOXRQqiji6dyK0+m7nguZFLwcDOSrM/5AmbO8CIbXtaXMvZ2NgR70PvX3fpF\nzWF8ZoIf1JqzfcDmHso92wLGl9e3MQIIk7aRBE3F60cfwUcplthe\n-----END RSA PRIVATE KEY-----\n",
"config": "apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n api-version: v1\n certificate-authority: /etc/kubernetes/ssl/kube-ca.pem\n server: \"https://127.0.0.1:6443\"\n name: \"local\"\ncontexts:\n- context:\n cluster: \"local\"\n user: \"kube-apiserver-requestheader-ca-local\"\n name: \"local\"\ncurrent-context: \"local\"\nusers:\n- name: \"kube-apiserver-requestheader-ca-local\"\n user:\n client-certificate: /etc/kubernetes/ssl/kube-apiserver-requestheader-ca.pem\n client-key: /etc/kubernetes/ssl/kube-apiserver-requestheader-ca-key.pem",
"name": "kube-apiserver-requestheader-ca",
"commonName": "system:kube-apiserver-requestheader-ca",
"ouName": "",
"envName": "KUBE_APISERVER_REQUESTHEADER_CA",
"path": "/etc/kubernetes/ssl/kube-apiserver-requestheader-ca.pem",
"keyEnvName": "KUBE_APISERVER_REQUESTHEADER_CA_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-apiserver-requestheader-ca-key.pem",
"configEnvName": "KUBECFG_KUBE_APISERVER_REQUESTHEADER_CA",
"configPath": "/etc/kubernetes/ssl/kubecfg-kube-apiserver-requestheader-ca.yaml"
},
"kube-ca": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIICwjCCAaqgAwIBAgIBADANBgkqhkiG9w0BAQsFADASMRAwDgYDVQQDEwdrdWJl\nLWNhMB4XDTE5MDcyMjA5MzI1NFoXDTI5MDcxOTA5MzI1NFowEjEQMA4GA1UEAxMH\na3ViZS1jYTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALsGg3vMA0RM\nnpv1Ejf4PX7YpAmR0DGejxS4cYkpYaf3EPpHXbIbOFmVS5etg1bn3ydaWU6g4+KQ\n09eCXDjETn0JV1mJVuLXhhm+Qkj9eiDb6QMiql1lO59aJWLOLwkvRCnZHI3ncIDW\nUcKt9Rl7Awd3TAfRHQSLuEuedJiDbtxjQQS3itVNJ3KhVdeahN9zC/MQiqRd88jl\n6EpFcJzwxDGGUEsCwqEtWsFUx+5/bjmqEpTmIHmiUZAmyBab1R0DnkrCaWWxZGZg\nRodBXrSDiVyA9mLXBbX7Mu+tMDHkXRcqfrTM7ALZB/a8V5bPBG+4AmoJQ4OeE+lM\nUm1HETn9EkUCAwEAAaMjMCEwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB/wQFMAMB\nAf8wDQYJKoZIhvcNAQELBQADggEBAF3/YggFQZGM0y7kMNBoYu80BZkrN32QOSo7\nASTG1XgapCJ68mmo5UwSYlDdCOcEA+YpA8pPTLNwSnGVBD6OqUhyfG1u4Lrz7OjT\norzEwNGexjRQBCfV45mvzHxK+BLplpLyZJ27vefF7AEHqL+Tg43jZUdX6n+rEoXo\nm1MEsazjwEFKEde+orqm3Wy0X0PsdHj4SFtIEMHyxZTx7+i22hHMH9LCAZUM+FdA\nv9t2scQljoYhAdlxrRnKtt6C+hetkS7bPVR0D+1/sq1L3LWYU0wKkCZ+UuL3yrap\n1JP2ES1oy2bUqB/1r9Q03fSRn9+snXh2i3vkskI2niHE8ivpDa0=\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAuwaDe8wDREyem/USN/g9ftikCZHQMZ6PFLhxiSlhp/cQ+kdd\nshs4WZVLl62DVuffJ1pZTqDj4pDT14JcOMROfQlXWYlW4teGGb5CSP16INvpAyKq\nXWU7n1olYs4vCS9EKdkcjedwgNZRwq31GXsDB3dMB9EdBIu4S550mINu3GNBBLeK\n1U0ncqFV15qE33ML8xCKpF3zyOXoSkVwnPDEMYZQSwLCoS1awVTH7n9uOaoSlOYg\neaJRkCbIFpvVHQOeSsJpZbFkZmBGh0FetIOJXID2YtcFtfsy760wMeRdFyp+tMzs\nAtkH9rxXls8Eb7gCaglDg54T6UxSbUcROf0SRQIDAQABAoIBAEyeuYUhKoFzplMc\nTq9xWrg5xtJI03NQkyuY+KKJuoFWfbUbQa5y6l4BBOwJ4NHcwpZbtFS7I0ExtigH\nNwLmoYUmK4v58QeT5lNrl7pRZhRKI6buL799drSs+L4G3Ew1+gXu9XzJQVT2+J36\nKHyc4bvQSuy05d9z4jfQndHRnrxYlarNkVEFjZGoVtcL39FhOecGfS9gsxi7aXCg\n69vHEyasnDG33WVehZEJkVwtabFbpl6PRbnUgGxVyk1Lq+cCGAE3KVxrOJPf7yzK\nihQp0UurqWuUQX6FGRE92gWAfdP60p+pMRuxj5o5fU+Ng0aHxZ22gUF/XEgCXqgo\nGvqtQi0CgYEAzZYGK+EMLbq9VT5ANgFO5PVI+QCn8+Yw82aJZ/wqdD6NstE3dCc6\n8xLKReYeMblOFIjM6tBedscaiReRBvlqVI5AnCjCitonFVDwo4CQOfOTe+Ttyop4\nSGn9uA5DkVgUrlgiIxU1TDcAmSOv0uvYGfrszjh5x6l+RqwPxXjKVCMCgYEA6ONS\nAx5oju+fWNSYYYZh6BQKI4+6IhUyxri/h0DMRbEqCQjmyOap3WSnkPUGviOI5PWN\nVM94BtQCxZ4LfWbAoOaNz2C4mzfCXAasMyw1i0sX+WhHmK4Q2uuHB5izeyAJ14vu\n09PhM5MtJVRwxlzR0tol0lf6sxkAZJNzgcV8kncCgYBlOeJuBpBlj9Ybu9wH+1rq\nV3AH0u/fos8lea5cegg5VneUvXktgwd+mhjok+b4GsK8T6N+Vvy8RUrmh6kMIxDc\n83ISsesgTUwDLeheyTW6oxlSXqVLHiUdmrqnvrY8QX9lXNXqKk4k6cwsmyq+kwvD\ntw6Ud7BWxDj9+UiGtfQBuwKBgQC3qEVN5lYomrn8ULPW/SYK4nrYbokoRbVkV9/b\nJvOw5jUzKMzvlFBTIwSuYT0vB1D21trdDjwmjl2dQLTo8PZqLUeaSjlrVv60miKE\nvJGBRvgiHOWRd3wdWO6ExByOtcZtMJJohMgtZxyMOtow5JQoBxcamX1AbjAMIOhP\nZ9GEowKBgDdNMQ7ZtZnoujW9k/aruXYbOyUdUbx9sXEQ5KdCQW9WgjiTvCGq+iDS\ngRL+GzRog7rqMclof0WzYw8a0Z47RhG2vGdTv6XP3YZBk8iI8MZGr4m5PS5O86O5\nDKaizhAvOMT2ugMhc/bFQEX3SuhQRgHSJfFRQsB1nPG63t/2panF\n-----END RSA PRIVATE KEY-----\n",
"config": "",
"name": "kube-ca",
"commonName": "system:kube-ca",
"ouName": "",
"envName": "KUBE_CA",
"path": "/etc/kubernetes/ssl/kube-ca.pem",
"keyEnvName": "KUBE_CA_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-ca-key.pem",
"configEnvName": "",
"configPath": ""
},
"kube-controller-manager": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIC5DCCAcygAwIBAgIIbj/24Ywoui0wDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE\nAxMHa3ViZS1jYTAeFw0xOTA3MjIwOTMyNTRaFw0yOTA3MTkwOTMyNTdaMCkxJzAl\nBgNVBAMTHnN5c3RlbTprdWJlLWNvbnRyb2xsZXItbWFuYWdlcjCCASIwDQYJKoZI\nhvcNAQEBBQADggEPADCCAQoCggEBANuQDBJys4d9baXF0ViExgA4UUE5DW7lrEwJ\n+BW0ODT5up092Cm6sEZQZKn5sobFXwh40A9kFq8Iq7SwsnsjeSmiOBphn8n2wfag\nV8468A9G44qUKXAGlZXWrxVwwZJWh5ITON8Q6bCVOsx7/ofWrmF1ER4y1LEaYzjY\nCX2xMJeHwRIv/VK7bTD8o+WVsslnX33ATrrLWHE6jHZYEH+cDQe6xlx5yRr3c+4R\ndxYT3yc9J1sIjR7PKZDDS4MyMwlkNU5LJ6HYjzw3MohRCZGMNRR96J4o0fZDonba\nTYwVwTL7OK0aaaoltFofsTp5DPJfi8dkGnhwwb4hRpinGN5eRzcCAwEAAaMnMCUw\nDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQMMAoGCCsGAQUFBwMCMA0GCSqGSIb3DQEB\nCwUAA4IBAQAJ4ahL5f1Ozp6HBdqnuz8WttUkj5pbdoLf4IOdzhDrhrhkw93UGywq\n04GWZS1nHZIFSfbP+RbIbTiYKlYAcLoqMFQW1SH8QmjmNjy2C/XI7wrWWZ9iYao0\njuLLBXX2V9W4QNtW4cL4UaJjslkw+1X+0gIxFsDWBPrSYyqeOpAvtZF7Hr88QQqE\ng7QR+KFzsPqSEhdBladPy+29hcMbTSgXYPvBzXvF7rCh7er+SY1IYYUVAQ8+AOPA\nqbkkslXERAo8iCKsBOMBMdg3MG1X/JqLnkyaXh/MhCRZaQCxplJaXJsOwDenF3Dg\ndXHsQL2ip4OpRQE7Fmy2SOQB/5GAibcL\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEogIBAAKCAQEA25AMEnKzh31tpcXRWITGADhRQTkNbuWsTAn4FbQ4NPm6nT3Y\nKbqwRlBkqfmyhsVfCHjQD2QWrwirtLCyeyN5KaI4GmGfyfbB9qBXzjrwD0bjipQp\ncAaVldavFXDBklaHkhM43xDpsJU6zHv+h9auYXURHjLUsRpjONgJfbEwl4fBEi/9\nUrttMPyj5ZWyyWdffcBOustYcTqMdlgQf5wNB7rGXHnJGvdz7hF3FhPfJz0nWwiN\nHs8pkMNLgzIzCWQ1TksnodiPPDcyiFEJkYw1FH3onijR9kOidtpNjBXBMvs4rRpp\nqiW0Wh+xOnkM8l+Lx2QaeHDBviFGmKcY3l5HNwIDAQABAoIBAFapyNoPRCWasIja\nSAN0hOLZV9kyt8aqCRQKa03F0Nm4KJygKI5wDdvzJ4if8oOXrCcf6J/qlbmI5p1v\nkUNhgRWZ3iMrnXKOX4kFSSMxyEd/Z5jvWlvJfYmjL4B6/SzR0rkptSz29AarIIZN\n+u2LI1F8b8FHtwQgskFhpH1kTxW/2JybMT5y4xCxBB4R6gIqxKDDhfbWo59t9dEI\nDaly3ulhYRv+VNEmKTn3hiY5UymzMlGHimFKLg6Q/DmLFTaV/DVHkmw1uoC7UWoV\nHVXNnmwtvmp3mF2rNXHGjVgRVw4oIU4yY0p8oz7gX5FFsvi82NFsDaraUTTGTN/K\nX2VYNGECgYEA6hKxpsqO4lsMzKEIHB1l4UxGBCV1qaTWy70F+W4QfFvl9zgmuI8h\ngVU1WGQMNgyb5dO4c4/r4hM0jV/Kl7M4bQy3ExLSinOkBDUw2JVyw8JDITRUiwpz\nw4M5L8T8m3Kd1A66VkRZ4rAfd2s2lVZVrrmg0RCK768Bylbgv7sO75sCgYEA8CFh\ndjlHU6hVXrL6DWNl7vos1b4uWZb5GM/TjEQ/wVHFbOB69XmOq4oqia2P3GCDw7fn\n1REebTVIpguUvyUh5dV47XNFw6QPGilqxr9szsPWrlBlNh/U6DR4eHtiT3ClwsUN\nrvXZTPA6v6v58arYZ9ujjFD6U0aCE6avmDNYlpUCgYA/qCvWH1oyZu6ZK+YHt1sA\npXyZHhLcuu6LY/Ge/vbOyzNEQ2oMYSiMgr7KNhM4E7/GBgkwrGBLEEwHcpinHt9m\nkMGtFz5Y1tAxMMIC9MnMQSV+fzvKwGHxNtuziQ/luor/o627TkVfp+Q21tp2TI9q\nwCLjGGugeWi0PWJL4jRSpwKBgGhKJxkAcXDs5/7T/nO4Gro6nyRJj7nLLu5OvJLr\nxUDBsdvCdsSMAaGQxT68SYGtzSxAgAhygeNuQ9oRp42Io/s8xmNiMAdzBjgXis88\nrhXyyhc0xXBHTOF4gGXbOGxGAUfg6zt0fObEYs+B8nC6AXjB1HsJZF6sfavpNdQF\nYNEtAoGAR//jxrQOZGi2i+COv+h6Lf9OUvf70Vu3lODqwQh/WBoCG/QyGZZcNPUB\nbPb0+XJEM38zCdDnH90xKtHQ9hyZQnbSvyMRi+xEExlcS5XSAOwlhGpiFzADOP0N\nc23a0TbR8YyKXQfPfxOokqE1vSxgcA/54qwTn3KujBHRrGQGIVY=\n-----END RSA PRIVATE KEY-----\n",
"config": "apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n api-version: v1\n certificate-authority: /etc/kubernetes/ssl/kube-ca.pem\n server: \"https://127.0.0.1:6443\"\n name: \"local\"\ncontexts:\n- context:\n cluster: \"local\"\n user: \"kube-controller-manager-local\"\n name: \"local\"\ncurrent-context: \"local\"\nusers:\n- name: \"kube-controller-manager-local\"\n user:\n client-certificate: /etc/kubernetes/ssl/kube-controller-manager.pem\n client-key: /etc/kubernetes/ssl/kube-controller-manager-key.pem",
"name": "kube-controller-manager",
"commonName": "system:kube-controller-manager",
"ouName": "",
"envName": "KUBE_CONTROLLER_MANAGER",
"path": "/etc/kubernetes/ssl/kube-controller-manager.pem",
"keyEnvName": "KUBE_CONTROLLER_MANAGER_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-controller-manager-key.pem",
"configEnvName": "KUBECFG_KUBE_CONTROLLER_MANAGER",
"configPath": "/etc/kubernetes/ssl/kubecfg-kube-controller-manager.yaml"
},
"kube-etcd-172-16-100-40": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIDejCCAmKgAwIBAgIIJtEZMxxRB0cwDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE\nAxMHa3ViZS1jYTAeFw0xOTA3MjIwOTMyNTRaFw0yOTA3MTkwOTMyNTdaMBQxEjAQ\nBgNVBAMTCWt1YmUtZXRjZDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB\nALgAF8Q0KQeet6qSHpNUGU0W2Vvhl8qO+Sd2TtCKijH2yo0XwYjsS41GzJv4aWUK\ngsZwWOhaDKXNDe5luF7onsGhe7eZlO1g3XOHvs1MwPaPo1Z90CbSNi6d89wkTjV4\nssemLjR8MezcS1ePOugqlxTeW46J9thfcGRRWQXNNbGj4IOMirY2oGeM+tDUdRll\nz8Q6rvjeWlR+5hMD8r1M5YsM88DtwFTaf7j2MSOf94ZMjRfI7ltRddLJErRfcNQW\noSEuykuMcFWEjotVx/2U/ioxnzRc/zvDYO40nTQiBL48JhdPeAt0xzXREGqqGYoD\ncWq+Ay9DfjiQETUoQkRf+zkCAwEAAaOB0TCBzjAOBgNVHQ8BAf8EBAMCBaAwHQYD\nVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMIGcBgNVHREEgZQwgZGCDm9uYXAt\nY29udHJvbC0xgglsb2NhbGhvc3SCCmt1YmVybmV0ZXOCEmt1YmVybmV0ZXMuZGVm\nYXVsdIIWa3ViZXJuZXRlcy5kZWZhdWx0LnN2Y4Ika3ViZXJuZXRlcy5kZWZhdWx0\nLnN2Yy5jbHVzdGVyLmxvY2FshwQKmgxZhwSsEGQohwR/AAABhwQKKwABMA0GCSqG\nSIb3DQEBCwUAA4IBAQCUJGa048YxZQYsBoN3d51do9SxtR+bdXRRuI+nhXzIwn2a\nC9YaSUmVlvJjIL0GBYqapiJ+++ASwAMmtGPtt2mIfyWV74TBHVLy1gYUQ/GEPTSm\nQ+W1RRwKGVx/sNmX17bkU5hW8dhs6fm3DQD+TYzvMxHNarvibeHbzVYc/g4XXj8q\n0EQQ9V+x0E+HcjI4nimuS2jfr301vE3QnUKlsYjKP5cONrchTL8KbCRq1JqUPXZH\n+ws/qtlLJbr9k/izcuq4KZW/y8WbDXI0eh1mUitnBRRGSVnnWo2mvSo9VevOBv7Q\nMvibGvuXYIryAKTQbEMQFnwYvYMri34GVCuptfFf\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAuAAXxDQpB563qpIek1QZTRbZW+GXyo75J3ZO0IqKMfbKjRfB\niOxLjUbMm/hpZQqCxnBY6FoMpc0N7mW4XuiewaF7t5mU7WDdc4e+zUzA9o+jVn3Q\nJtI2Lp3z3CRONXiyx6YuNHwx7NxLV4866CqXFN5bjon22F9wZFFZBc01saPgg4yK\ntjagZ4z60NR1GWXPxDqu+N5aVH7mEwPyvUzliwzzwO3AVNp/uPYxI5/3hkyNF8ju\nW1F10skStF9w1BahIS7KS4xwVYSOi1XH/ZT+KjGfNFz/O8Ng7jSdNCIEvjwmF094\nC3THNdEQaqoZigNxar4DL0N+OJARNShCRF/7OQIDAQABAoIBAFgpKzoGJ/aPtL7U\nfjPnIqxOG10rzRfvwy+GPyMqjiZ8cfaCgGeDmHubhaPhtVYFYD2GTQMfSloS0u77\n7AgFezOZlPaWbsYCGHknfGNPuFEq538kjxxasj1JzzMWbiauqzJ/zDm3nBA6SRst\n9HKTHYqRM9sq/YwUxnw6geYGughfhNvKaLWiS2YrsD2gPuq75GV+wyK5n7yq7mjX\nyGf5+o8Vuw4dLCMRfOWYpQ32JURnXAqQ3ZJ47oiOgv1yJIVaE7ci7eKtcj+jnl8U\nPbyWuAwRZNCU1lRqzzZrhnJcDmVZWpU/N6iqY/01v5XfZ5vOqowwFzU4T57v0e9m\nMBMeIRECgYEAxRcoaeHwhH7JCXhmxAWggYHNMAVM/lt4isLVn3hIELkTrom6Ldm0\nE85B3LcJLxkkdep2LIyL4yNtipp+eJyCjfZwV5dHwt5acKHowN5je4iige5us/cw\nMopLybfxw7IhLDUBp/1hWFzN6S1099FxQkdBAwXUWkiXM1s3q0/7GYUCgYEA7v9Q\nsk3QGYqLhfsu40PMnhYupoUs4f23HNuhNbO6Zt2QecKspP5go7yttF4Exqk0HsJz\nliO778Jipw6P1+E8yMwSE8tzmsdtkkja9YWR809cYbK9BTf2MphbejDH60U5fesx\nIfLpOLvksmH502OWBWgYbUYYL5wioioQB8jEjyUCgYEAj3IqbN4QIy7v/R+PlC4Q\nQxIJGGFTR3Gly0DBIF73/IvnwknlosdnpF6jWxQw8AGZouTx+7yTd87qRE3rrLck\n4lQyMd4eUbtqEb4V3II/38JznKPAGOXZQ3+aZ4ekw+0a7cTqnqzxM9Hf4E3AazoZ\n98ZKj/unMcrRmYRTgTEmLc0CgYAv4NQAtFfxFfE3iQduneF69gSodVt1doddb55P\nDM9V1iMgUWCWuSMRCjZeQMuCJNl9JAVV2ZjHwlBTGTa1z7iGnWMttf9VzYfk21WK\nBBUmAgsGw3Dcf8aBgkYpJ2aVeku3LwurumcAZ5u8JKaQs42VTbI/CeDEgOnrRLPr\na0v9RQKBgGS6+NF9q/d1X2ZAl60j5msUzVn15/BUO8o8IthMcJtfgKbPuuxeWcLl\ndu7BOeegHNx5pOjCpWKPRZlNbQ0Txbhtd0qaEw3OFg/WIpH31wFmVb3AtSeWMMmG\ngqk9Hw01Oa/zvK1j6cCuwxHAvXkHkjvfbYeXDU/M2y7j0uO1dO8u\n-----END RSA PRIVATE KEY-----\n",
"config": "",
"name": "kube-etcd-172-16-100-40",
"commonName": "system:kube-etcd-172-16-100-40",
"ouName": "",
"envName": "KUBE_ETCD_172_16_100_40",
"path": "/etc/kubernetes/ssl/kube-etcd-172-16-100-40.pem",
"keyEnvName": "KUBE_ETCD_172_16_100_40_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-etcd-172-16-100-40-key.pem",
"configEnvName": "",
"configPath": ""
},
"kube-node": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIC6DCCAdCgAwIBAgIIB0/oCi9Y/K0wDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE\nAxMHa3ViZS1jYTAeFw0xOTA3MjIwOTMyNTRaFw0yOTA3MTkwOTMyNTVaMC0xFTAT\nBgNVBAoTDHN5c3RlbTpub2RlczEUMBIGA1UEAxMLc3lzdGVtOm5vZGUwggEiMA0G\nCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCwzeNl3n1og5F5XxQfXs3bBjeB4QZo\n2RoMZjSrV2aR66wziFVLuoqVfstOedwTIv3HlkPYRgZuMD5fd20nP5NCG3skXf56\nSP/C87ye2zVg1FOXAwuPVQtBrWkEbR/ftCenYfnpbtcTYnxf1sIesEQtNKhGt9n8\niBuugv5wQD/+KNenXEAkyKNScViVZqfg9OODJtc0YJvrX1eGZjE73GEhEO5Ndd0A\nHWHKSpMnTHDaQFnPgy8aoybtdVNLFtz7g8FxzaGkT779WJ09tkAxjf+g+dPXQy0u\nF5aRwhWquCZPbVOkGd/tM99dcxrNxHT2oqiOHIoGj5LVEyX0FgDgmTvXAgMBAAGj\nJzAlMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDAjANBgkqhkiG\n9w0BAQsFAAOCAQEAUy/VyWVjiHxd/y03ZClnzNA6PxTiOpQgwlUpP8YekJd3JTmK\nqryVL1Djp+UKYozMV9Crp3RLprGphmrzrlQjmU1sN1kRbaXKAcic5cMxAbwz+dos\nrLulUCW7YSwG3cTbWbaDVN3IykVtyWLbnB6D6JxwReoyKf6O3lrIR/9Bi7rpUhg9\n8q9hMecpuYtnIDIWIX4mgBHwgIAaVYwV60tvi6OFS+t4OFpzx92mJd0V8IxZktOd\nVgddAN0fHsSpW+j1j+L+bBwIW48pulKfKpPEGAi9m1uPW+rNz6MgU+HrSIKzA4Os\nqKgPOejfeyZB+19Sez3S+8bn5fO/m0BsMLi6hA==\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAsM3jZd59aIOReV8UH17N2wY3geEGaNkaDGY0q1dmkeusM4hV\nS7qKlX7LTnncEyL9x5ZD2EYGbjA+X3dtJz+TQht7JF3+ekj/wvO8nts1YNRTlwML\nj1ULQa1pBG0f37Qnp2H56W7XE2J8X9bCHrBELTSoRrfZ/IgbroL+cEA//ijXp1xA\nJMijUnFYlWan4PTjgybXNGCb619XhmYxO9xhIRDuTXXdAB1hykqTJ0xw2kBZz4Mv\nGqMm7XVTSxbc+4PBcc2hpE++/VidPbZAMY3/oPnT10MtLheWkcIVqrgmT21TpBnf\n7TPfXXMazcR09qKojhyKBo+S1RMl9BYA4Jk71wIDAQABAoIBAHKuugfDz7jRTrVZ\nlOXcSs79M1gNrGiqGhlc2mEhqiiQo/wxPJKIh4f83MckiDnpgo/Jymwoz9b9eBfY\nkXbZ6B0DUlb+mSic3TmJfBnWusL6NtZHZWItmcWaQ/pZBknGw8nw5XgjK2MpiN9p\noDTjxPVFqLlmXAFQEBScZ4pMQrYKl9X8h8ccWtfEqukKMbNpjRHGs4HO1qidVFZk\n0yWeS9rH1ckirGpJTXjV9HZr4Q0hbkwtWb87syCmyb5I0AqYm7TpKbYUJ5coDbDZ\nE00uOKVlsXd5f4wQSJxMB+8U6JOoo8tE4PALpsgwAEhoPDRUG88JFHIQh0IoQP63\nf08SmUECgYEAzOTyaATl2py15gwDg7WhorDh1staXSBk3XrMaO8v27vTLUvy4FCk\ncgWyLur08lq4LhXMcsgUJpPFXouo/Ir6gXWnEL0+rTrS8qssKQpIm1HrPClM4ToL\n3pAPm++EHvIydOA8gv83QGSr3+QO1p3BOSrVZSJDCSIH/oxaAll5bWcCgYEA3OdQ\nk63sMNsGKmroopBO+X/gWjJm2WVWWCePdpnSDdaH9HohvqPNog7KiS9kZYxqlOLM\nfOsNyeudnD2HZX7mpq1KuBMykPy1veLMKgZw/jM+rgz5IgrDH8nItS/YsxAwEd6J\n4hvulaBecUpG5G3cGUPDzhHz6JRk2jU63jKcSBECgYBiaNA4v6l6f2wqSrjhaLpy\nfoxp8w+6zNVFPzawa7lLy4GZLvI/ZNwm26+CM11kjcgKcnPY6VosUSspxuBqM4Iv\n7cdJFevlnew1m5tBNuw6lXTrV1bSkmOjCQRGPBr0FhW/5veJX3U1G2Ulys6JrAlj\nILyPr44M8+htDRIqMLlOnwKBgQDJg/ackbd6JgOs5FQOIEj7D+2ROpNaVDh9OLGG\nBvJ5DzB9FID+ER3AbW/Tz7jtbje4VGEov6CXh+oCopqGO0qb2E/mNhUV8IK2k1Oe\nLrwbGJJgYpNOu7SepgwDRX+Qcf73/iOYcdSnCGG/RXwAXDvBjHDt5rCzhdOuv4pR\nMqggkQKBgQCHW9VVIyPFSlC+apRJ3lLqCyvalqK/3E7GsNCDQqEz9rVOJJBkHZ0q\njkTiGSjy8/H8roIV0nVtUcXBpKf7vlOdDEEEvp9K/R3KvVtJ625SZQFlcpVrjQ/M\nihuZ8or3m7Fh52FiHYGoBg8XFrIYrRSqk5qNHoyOodXnUoEL7lQ9kA==\n-----END RSA PRIVATE KEY-----\n",
"config": "apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n api-version: v1\n certificate-authority: /etc/kubernetes/ssl/kube-ca.pem\n server: \"https://127.0.0.1:6443\"\n name: \"local\"\ncontexts:\n- context:\n cluster: \"local\"\n user: \"kube-node-local\"\n name: \"local\"\ncurrent-context: \"local\"\nusers:\n- name: \"kube-node-local\"\n user:\n client-certificate: /etc/kubernetes/ssl/kube-node.pem\n client-key: /etc/kubernetes/ssl/kube-node-key.pem",
"name": "kube-node",
"commonName": "system:node",
"ouName": "system:nodes",
"envName": "KUBE_NODE",
"path": "/etc/kubernetes/ssl/kube-node.pem",
"keyEnvName": "KUBE_NODE_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-node-key.pem",
"configEnvName": "KUBECFG_KUBE_NODE",
"configPath": "/etc/kubernetes/ssl/kubecfg-kube-node.yaml"
},
"kube-proxy": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIC1zCCAb+gAwIBAgIINJA/Jv+EeX0wDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE\nAxMHa3ViZS1jYTAeFw0xOTA3MjIwOTMyNTRaFw0yOTA3MTkwOTMyNTVaMBwxGjAY\nBgNVBAMTEXN5c3RlbTprdWJlLXByb3h5MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A\nMIIBCgKCAQEAsM3jZd59aIOReV8UH17N2wY3geEGaNkaDGY0q1dmkeusM4hVS7qK\nlX7LTnncEyL9x5ZD2EYGbjA+X3dtJz+TQht7JF3+ekj/wvO8nts1YNRTlwMLj1UL\nQa1pBG0f37Qnp2H56W7XE2J8X9bCHrBELTSoRrfZ/IgbroL+cEA//ijXp1xAJMij\nUnFYlWan4PTjgybXNGCb619XhmYxO9xhIRDuTXXdAB1hykqTJ0xw2kBZz4MvGqMm\n7XVTSxbc+4PBcc2hpE++/VidPbZAMY3/oPnT10MtLheWkcIVqrgmT21TpBnf7TPf\nXXMazcR09qKojhyKBo+S1RMl9BYA4Jk71wIDAQABoycwJTAOBgNVHQ8BAf8EBAMC\nBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQADggEBAHe6iD37\ngrfR0pHNJF5RDdt1t4tZv3LfJSsXN/Lrf1PSkCCPQBfGXbTxKxNC05hgUdoh85/f\nR9RhFqROZWqcnTOrNqkN+DohL2HyU3pdQWTy3ivzAiTjl4Cx9A8bY550yqsPHLkQ\n3dY4vfPMwHZ0S4pjFqnVVYx+4UxbeC4Kwd9qdUpPC6eDC7qY1kM4jNYz8ep5pMF2\nk7a7U1b5U2RcV9afKcD4Zml6fEgfuD1IaBWBfgNYxUJOFg6QAfmK1a4HTr91k4NI\nCQtxGTGwktKpybi6h6/gGplKX8vDmg7YpieM3dyWFwci9sPmPFImSQ5HedT1NAOr\nVfol7n27ZeKj0Q0=\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAsM3jZd59aIOReV8UH17N2wY3geEGaNkaDGY0q1dmkeusM4hV\nS7qKlX7LTnncEyL9x5ZD2EYGbjA+X3dtJz+TQht7JF3+ekj/wvO8nts1YNRTlwML\nj1ULQa1pBG0f37Qnp2H56W7XE2J8X9bCHrBELTSoRrfZ/IgbroL+cEA//ijXp1xA\nJMijUnFYlWan4PTjgybXNGCb619XhmYxO9xhIRDuTXXdAB1hykqTJ0xw2kBZz4Mv\nGqMm7XVTSxbc+4PBcc2hpE++/VidPbZAMY3/oPnT10MtLheWkcIVqrgmT21TpBnf\n7TPfXXMazcR09qKojhyKBo+S1RMl9BYA4Jk71wIDAQABAoIBAHKuugfDz7jRTrVZ\nlOXcSs79M1gNrGiqGhlc2mEhqiiQo/wxPJKIh4f83MckiDnpgo/Jymwoz9b9eBfY\nkXbZ6B0DUlb+mSic3TmJfBnWusL6NtZHZWItmcWaQ/pZBknGw8nw5XgjK2MpiN9p\noDTjxPVFqLlmXAFQEBScZ4pMQrYKl9X8h8ccWtfEqukKMbNpjRHGs4HO1qidVFZk\n0yWeS9rH1ckirGpJTXjV9HZr4Q0hbkwtWb87syCmyb5I0AqYm7TpKbYUJ5coDbDZ\nE00uOKVlsXd5f4wQSJxMB+8U6JOoo8tE4PALpsgwAEhoPDRUG88JFHIQh0IoQP63\nf08SmUECgYEAzOTyaATl2py15gwDg7WhorDh1staXSBk3XrMaO8v27vTLUvy4FCk\ncgWyLur08lq4LhXMcsgUJpPFXouo/Ir6gXWnEL0+rTrS8qssKQpIm1HrPClM4ToL\n3pAPm++EHvIydOA8gv83QGSr3+QO1p3BOSrVZSJDCSIH/oxaAll5bWcCgYEA3OdQ\nk63sMNsGKmroopBO+X/gWjJm2WVWWCePdpnSDdaH9HohvqPNog7KiS9kZYxqlOLM\nfOsNyeudnD2HZX7mpq1KuBMykPy1veLMKgZw/jM+rgz5IgrDH8nItS/YsxAwEd6J\n4hvulaBecUpG5G3cGUPDzhHz6JRk2jU63jKcSBECgYBiaNA4v6l6f2wqSrjhaLpy\nfoxp8w+6zNVFPzawa7lLy4GZLvI/ZNwm26+CM11kjcgKcnPY6VosUSspxuBqM4Iv\n7cdJFevlnew1m5tBNuw6lXTrV1bSkmOjCQRGPBr0FhW/5veJX3U1G2Ulys6JrAlj\nILyPr44M8+htDRIqMLlOnwKBgQDJg/ackbd6JgOs5FQOIEj7D+2ROpNaVDh9OLGG\nBvJ5DzB9FID+ER3AbW/Tz7jtbje4VGEov6CXh+oCopqGO0qb2E/mNhUV8IK2k1Oe\nLrwbGJJgYpNOu7SepgwDRX+Qcf73/iOYcdSnCGG/RXwAXDvBjHDt5rCzhdOuv4pR\nMqggkQKBgQCHW9VVIyPFSlC+apRJ3lLqCyvalqK/3E7GsNCDQqEz9rVOJJBkHZ0q\njkTiGSjy8/H8roIV0nVtUcXBpKf7vlOdDEEEvp9K/R3KvVtJ625SZQFlcpVrjQ/M\nihuZ8or3m7Fh52FiHYGoBg8XFrIYrRSqk5qNHoyOodXnUoEL7lQ9kA==\n-----END RSA PRIVATE KEY-----\n",
"config": "apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n api-version: v1\n certificate-authority: /etc/kubernetes/ssl/kube-ca.pem\n server: \"https://127.0.0.1:6443\"\n name: \"local\"\ncontexts:\n- context:\n cluster: \"local\"\n user: \"kube-proxy-local\"\n name: \"local\"\ncurrent-context: \"local\"\nusers:\n- name: \"kube-proxy-local\"\n user:\n client-certificate: /etc/kubernetes/ssl/kube-proxy.pem\n client-key: /etc/kubernetes/ssl/kube-proxy-key.pem",
"name": "kube-proxy",
"commonName": "system:kube-proxy",
"ouName": "",
"envName": "KUBE_PROXY",
"path": "/etc/kubernetes/ssl/kube-proxy.pem",
"keyEnvName": "KUBE_PROXY_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-proxy-key.pem",
"configEnvName": "KUBECFG_KUBE_PROXY",
"configPath": "/etc/kubernetes/ssl/kubecfg-kube-proxy.yaml"
},
"kube-scheduler": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIC2zCCAcOgAwIBAgIISyUOSiasjpgwDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE\nAxMHa3ViZS1jYTAeFw0xOTA3MjIwOTMyNTRaFw0yOTA3MTkwOTMyNTRaMCAxHjAc\nBgNVBAMTFXN5c3RlbTprdWJlLXNjaGVkdWxlcjCCASIwDQYJKoZIhvcNAQEBBQAD\nggEPADCCAQoCggEBAL/2OnG7uls+YaSc624oX3IK8Rr6YqVRjFVhrzQqEX+j9uoQ\nXUDpkVIlBJbh55wmXRAZABClv0M57XYgb1hTdBRM4ecURoUw3CVTdmz3XSXA4QJx\n7yPPv5ERzhe3EVH6oTHi9T5EjU/oezp8WigZHhw3I7/s0TpoGtWPJmtOOTJpaw8X\nYXAsOosirY2RiBN+xj3tcnS24UXdtmTjV3eRJCsYdU8JMzX7zmgES0pSdbo8cAYG\nAozw2W1rXTP7QoGO2XmxNZMuSjD1EYKaAEHAYESg/jUh56rBxx08Q+huFdrmVTZY\nTzCAN0p2gv6R/SPPZ6Ov/01AmbfuaYCEynEuG78CAwEAAaMnMCUwDgYDVR0PAQH/\nBAQDAgWgMBMGA1UdJQQMMAoGCCsGAQUFBwMCMA0GCSqGSIb3DQEBCwUAA4IBAQC2\nTd8cf/8zPBeOcnOcPl/fAy+34Q6r5kr8iyKlCI/FS3AxAAafHPN7xGPDAchV7iz4\nZSm5lEJHGow/iG/LN+1fA2C8T5PHfRlN4sM1HfGKNwwDgAoQvfVW60A4paMKgeNy\nAyYnz97rRp5sHlzMagn4+yS4x9niIOWpdjDj+73IkglsKGkL0DnXKCcfowcaX7sK\nXys3rDMdecAW4+TQvIjjgIyOI6Kcrg/ewfsPKLlzMi1KhWEnsPvcIuB+rTVysVKo\nh7obB5E8jSte/QG1RCOr9GHy1usxUweujtAlCbwuXdvcM8TWEW4tPMA2nBdJ0Edi\nt7Ajr00MHerPHQn/qs+q\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpQIBAAKCAQEAv/Y6cbu6Wz5hpJzrbihfcgrxGvpipVGMVWGvNCoRf6P26hBd\nQOmRUiUEluHnnCZdEBkAEKW/QzntdiBvWFN0FEzh5xRGhTDcJVN2bPddJcDhAnHv\nI8+/kRHOF7cRUfqhMeL1PkSNT+h7OnxaKBkeHDcjv+zROmga1Y8ma045MmlrDxdh\ncCw6iyKtjZGIE37GPe1ydLbhRd22ZONXd5EkKxh1TwkzNfvOaARLSlJ1ujxwBgYC\njPDZbWtdM/tCgY7ZebE1ky5KMPURgpoAQcBgRKD+NSHnqsHHHTxD6G4V2uZVNlhP\nMIA3SnaC/pH9I89no6//TUCZt+5pgITKcS4bvwIDAQABAoIBADgya3W0UaA4qPk5\n/ypmo1qNnXwaZRqkvwKX8GTDFBBb/gMT+1jvh92WfLt3Mwjhqkj2KLC29UFHCh+U\n60PeWpM+1ruBkguX/DrpLjhESn6s8rJ2jF3dieV8YutKasWnK92g5c7u3rfQBZNf\nqD0o7UmxUsPzHmWM6J4wpDyxz8jT1Z6Jxf3dDJyYQLbOTMVFG0SglAwMFgzzteCY\npqln7MSKeBrcRwdO7ORfXMPhjJCPaVwR+9mQ9iEQ4tOY3n2BMWjs0biiiZ0SRmJv\nRTSCp85RKlhp06vjx+Mu9HQPOA2abiIbiXUWUChjcsFEBcR9vPx7e72V9hPpeNs0\ny9nUsSECgYEAzdbKTotlIyCZn6SL4t8dRrxg3SV0PbU/PI2nwu+U33EpsJ+7OKD0\n9BLEKlwT9xbInqh20OLWHpyEvNd3vvQBl0g12K1u9lb4bCkQhmb2Y8eVmYd7EEYt\n0HdPvpUPZWtSiy3P2BqYdIwC9Tq0qahAHRK/jOQY3P+O+N+7SxQPOpkCgYEA7r22\njpa09xqgJaWSm2qiqI2IhJiDDtRMWgbcRGsu6pndu+MmkJ33nMcJaCp/SFuBxpdX\n9G0CUOX+/gXyr+Z5ZFLBpUA1uwZ5YlTYSPEgjUY+s6zEzws9egmqWBRC9fgLL7ob\nYMtY64LQKoKGy8AK/Qo/g9KRxAHYgTDsamFNmBcCgYEAvH084rH5GjKnGq5GPqlL\nGNQ16yx6BmvrjVq2TeawL/BjWc7bVZ7m0H5CzftlePWPHuJrcxrVMlDnqDmy+NRe\nhJbmYBVpMAEyGXF9JWYxp+tCzcVnZt6s42HuTzoWUCv3+WUpF946zzbLGp74gweE\nNltNtwnEztLirHnL+5NR4rkCgYEAlti1/ZhJ1Yw0PjVE0gBpsDhlVuqKfJc1evi0\numJC7aa0zJ+04VF+f8ffMLsRW2W+NP2xbFZoTTdVi6m0cuIcN4B2rHRKvtZTYgcb\n9dKtb7Y/rgVvAScn+/EZeberq4DDiK991wYFNCGeaA27IAhCqSmNgjxNL9y+sLp5\n1qmOfpcCgYEAsa48sgQNpIF/gj13cdYWI/+YcJP2J974tWTvSqEVqrtU8pSiunPL\nvXeHq8j/fpySia4nrVwj9Iai6ZNvU0D1spo1jdL6WntgbFVBlvcN7IYMm0J05nVH\nfCaeLegImvIKlsQHWKRSLT7TzAB+126l2FlCtRjhSkdtcrBOd6HHz8Y=\n-----END RSA PRIVATE KEY-----\n",
"config": "apiVersion: v1\nkind: Config\nclusters:\n- cluster:\n api-version: v1\n certificate-authority: /etc/kubernetes/ssl/kube-ca.pem\n server: \"https://127.0.0.1:6443\"\n name: \"local\"\ncontexts:\n- context:\n cluster: \"local\"\n user: \"kube-scheduler-local\"\n name: \"local\"\ncurrent-context: \"local\"\nusers:\n- name: \"kube-scheduler-local\"\n user:\n client-certificate: /etc/kubernetes/ssl/kube-scheduler.pem\n client-key: /etc/kubernetes/ssl/kube-scheduler-key.pem",
"name": "kube-scheduler",
"commonName": "system:kube-scheduler",
"ouName": "",
"envName": "KUBE_SCHEDULER",
"path": "/etc/kubernetes/ssl/kube-scheduler.pem",
"keyEnvName": "KUBE_SCHEDULER_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-scheduler-key.pem",
"configEnvName": "KUBECFG_KUBE_SCHEDULER",
"configPath": "/etc/kubernetes/ssl/kubecfg-kube-scheduler.yaml"
},
"kube-service-account-token": {
"certificatePEM": "-----BEGIN CERTIFICATE-----\nMIIC4DCCAcigAwIBAgIIVP8yf80JgeEwDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE\nAxMHa3ViZS1jYTAeFw0xOTA3MjIwOTMyNTRaFw0yOTA3MTkwOTMyNThaMCUxIzAh\nBgNVBAMTGmt1YmUtc2VydmljZS1hY2NvdW50LXRva2VuMIIBIjANBgkqhkiG9w0B\nAQEFAAOCAQ8AMIIBCgKCAQEAzjkwbuLbDHMgC85PXGCU5hMhgb9RHEP3h6/HcwY7\nwfAiHHUFe895r3PCFAzUJdCtrKKFMYCQu0Nirl4Khv0e13JjZ9KxZ7/n/JQt9+qP\nENnJaunNqPu7tDWXc4Y71EaNJOkDyBSl9kzyHEXc4xYLi8CCt2EV7rSfYuRAb4T3\nfi2eAo1J+E7jEfRqoMTQ2YJdZ29+/zruo+W7FY1wRSy9QIoOfSafIqvUzoG+G16N\nwCJVu7IYO73yUzCKvHBOqMVCFyXMHVCRTFJgIHg6JCSP3oJYFEDfKaj6Ag6bGddl\nIsBfZ9/2xx+69I2M0/A6vtF4T/Z5FT+H3iikMpWu4FYV8QIDAQABoycwJTAOBgNV\nHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYIKwYBBQUHAwIwDQYJKoZIhvcNAQELBQAD\nggEBAEM2+oegi+2/+6C7j5JziNEmqzok0DyijSUnZNpxNskPRrFOznxXWW+NYsW3\nm5v9ppEKHBwB1tf18CS06zsAGE52j0WK/s4AZnxHOXaf3rjAF4264dCYe7PZnWJ/\nqRC4uN5//p17NQ10xuXBHLVJHjYQ9UdNP8ijqJRhoSiI89VTXsUm4Lh3Qn+c/INm\nayrOHoJU9WZu2ObiacidRX6hYaSqBt2Toc8oiPpZk8oeOK47efnvlUx9S1qSftup\nzMViTscLbmXWYgiIgIBTy70cy0VOkm0+h8e/zpzFXvtcxyBbS4xnHKZRIczbuq8h\n2huwsZeybmDXsGIo+VW/TRXvJ7Q=\n-----END CERTIFICATE-----\n",
"keyPEM": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAzjkwbuLbDHMgC85PXGCU5hMhgb9RHEP3h6/HcwY7wfAiHHUF\ne895r3PCFAzUJdCtrKKFMYCQu0Nirl4Khv0e13JjZ9KxZ7/n/JQt9+qPENnJaunN\nqPu7tDWXc4Y71EaNJOkDyBSl9kzyHEXc4xYLi8CCt2EV7rSfYuRAb4T3fi2eAo1J\n+E7jEfRqoMTQ2YJdZ29+/zruo+W7FY1wRSy9QIoOfSafIqvUzoG+G16NwCJVu7IY\nO73yUzCKvHBOqMVCFyXMHVCRTFJgIHg6JCSP3oJYFEDfKaj6Ag6bGddlIsBfZ9/2\nxx+69I2M0/A6vtF4T/Z5FT+H3iikMpWu4FYV8QIDAQABAoIBAQDHs0nbl4NYAwQ9\naoQfiMeoJU6FPqdlH8xMcURLdvXXUJs3w3qT8s57wTVoBSRLmlEUpC0nSsDdFtww\nzpCTN/rt16+7kqSJO6WDSltZgKMaRkxuZArsYRnb4SnQjWTSo5965srqrlQot0qd\nZmoI3ghARSRQQIvkpLTUJwTe+XxRik44omEgkYzOW6/gahsXee0dBVyN9a+KKaLV\nv3kLvrjhiPn6bYXykwxRsxSzksIflkDpJoo0DogB+7ZK49Bcf0lYYhh4RRVGrkQ2\nOwnqopottu4sVLgCcMN7BR2gSLS6gudXxfqlkDO/e4lkBzPdWHypxOBQH81S3YFF\n1g1MWc2BAoGBAOhVjXZrsiv+fXmD2vW9RCIpzlWRPFiJ4E2UIshhug4LC/ZNoPj4\nNs1ooT9qZUgCpv2pFl0CwIOrBmXpcNjZalshTH1D4pgIAwB19YPeameukt56u/Zt\nVTpGYkmpus36WrPtNVfAMcnR0uaCYWT9hzwd16LvGr1DlWpwpDDAoQ5NAoGBAOM6\nwppB9dLs9tYFirWCU3Jc0yykrvxVnqB2d7rqOcX1XTJoWh1vtIJL1lYRh02ayi2j\nwx6rScywu8dHQlfk6b00NqFSlP+TnbbXmF1gjsxxDWf3iwXhDC6Q8ePkEMJjEONs\nzRN55ZxlLSxe+uBSQ0G0dC7W+/rWS8M3JtKhRqA1AoGBAOX/e5FVwAZJYxGedG59\nNSvU0wxmQKUtqL1j/qL8C+HHEcp8wvR1VFs4TximTSliZsoKT7c3+DjgX+8Xj708\nTkt0xXHkeDclB0O3gptO3WtSqNZSDi0/wvVzo+1kdyddxdkE5J/ABgnTC8NiIuSF\nDxF2QXhs172xYAMFC6WvrECpAoGAKxcdiDsx7snZFUT/xOPuByXrNfAU7u0do7hT\nhDKeuheJkOGRFTv1sy/GeZPhYKoV5QnFGq/I0vRA581vm1Nhr+uS4IzUjJLluaKw\n1iRVcBeQP0MGLhs7QClkdZ7uPq6iqT4ctbDMBhh5tIxebuOqtW8PCULjjfHNdDul\nNRQ4XHECgYBA409pgXoxwAd0HoxLMeyPYdKnoc0Rvqcz0qP1a25xt+71VzVHkUQb\nRFksi3GYRbHVG5Ct07PjOBkpg8Xruj5og263lk4zNOketZUvvn/9byYvdsSxRBA8\ng2j54okDWy9T4taqR8tHI26gewizD1A2zn2gOEjsnfAgikPiuan8fA==\n-----END RSA PRIVATE KEY-----\n",
"config": "",
"name": "kube-service-account-token",
"commonName": "kube-service-account-token",
"ouName": "",
"envName": "KUBE_SERVICE_ACCOUNT_TOKEN",
"path": "/etc/kubernetes/ssl/kube-service-account-token.pem",
"keyEnvName": "KUBE_SERVICE_ACCOUNT_TOKEN_KEY",
"keyPath": "/etc/kubernetes/ssl/kube-service-account-token-key.pem",
"configEnvName": "",
"configPath": ""
}
}
},
"currentState": {}
}
INFO[0000] Successfully Deployed state file at [./cluster.rkestate]
DEBU[0000] Host: 10.154.12.89 has role: controlplane
DEBU[0000] Host: 10.154.12.89 has role: etcd
DEBU[0000] Host: 10.154.12.87 has role: worker
DEBU[0000] Host: 10.154.12.73 has role: worker
DEBU[0000] Host: 10.154.12.81 has role: worker
DEBU[0000] Host: 10.154.12.80 has role: worker
DEBU[0000] Host: 10.154.12.58 has role: worker
DEBU[0000] Host: 10.154.12.59 has role: worker
DEBU[0000] Host: 10.154.12.68 has role: worker
DEBU[0000] Host: 10.154.12.102 has role: worker
DEBU[0000] Host: 10.154.12.77 has role: worker
DEBU[0000] Host: 10.154.12.65 has role: worker
DEBU[0000] Host: 10.154.12.69 has role: worker
DEBU[0000] Host: 10.154.12.103 has role: worker
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [10.154.12.73]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.102]
DEBU[0000] Connecting to Docker API for host [10.154.12.102]
DEBU[0000] Connecting to Docker API for host [10.154.12.73]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.87]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.65]
DEBU[0000] Connecting to Docker API for host [10.154.12.87]
DEBU[0000] Connecting to Docker API for host [10.154.12.65]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.103]
DEBU[0000] Connecting to Docker API for host [10.154.12.103]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.81]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.77]
DEBU[0000] Connecting to Docker API for host [10.154.12.77]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.59]
DEBU[0000] Connecting to Docker API for host [10.154.12.59]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.58]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.69]
DEBU[0000] Connecting to Docker API for host [10.154.12.69]
DEBU[0000] Connecting to Docker API for host [10.154.12.58]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.89]
DEBU[0000] Connecting to Docker API for host [10.154.12.81]
DEBU[0000] Connecting to Docker API for host [10.154.12.89]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.68]
DEBU[0000] Connecting to Docker API for host [10.154.12.68]
INFO[0000] [dialer] Setup tunnel for host [10.154.12.80]
DEBU[0000] Connecting to Docker API for host [10.154.12.80]
DEBU[0000] Docker Info found: types.Info{ID:"RO7R:AQGE:ZDVI:2PQB:NXI6:RV6V:FVJZ:RL3Z:76HO:GPYW:OLZK:TWX5", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.812381091Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc000234230), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-12", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"ZB6A:ZLVJ:FWWV:UOUB:N55A:OJLW:ZUCH:KT4D:VXNA:RPRW:SIVX:MPGT", Containers:2, ContainersRunning:2, ContainersPaused:0, ContainersStopped:0, Images:6, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:32, OomKillDisable:true, NGoroutines:45, SystemTime:"2019-07-22T06:48:12.215813812Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001a0230), NCPU:4, MemTotal:8363880448, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-control-1", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"IEG6:SDXC:7QUA:HCPW:W7L3:YFNL:LO3O:H6D7:2ZWJ:E5OR:73UN:CYSH", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.994761137Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001a0380), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-8", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"FY7Z:SWU2:WBAE:EK3U:KQFM:CYOM:BOVV:MAFC:5TKY:OKWG:3W43:DNIE", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.92184308Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001422a0), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-3", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"4C3X:3TLT:OO2V:R656:MPWO:V426:JW6A:5X4T:WLDR:4B2J:HZKV:XJO6", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.875740964Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001423f0), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-10", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"HEZP:RWUT:GZKB:NDRK:SBZR:U4QY:6CHV:AAEF:2GVU:V4EL:ZBA6:KVJ7", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.976727581Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc000142690), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-7", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"B52N:ETSW:WEH7:A3DE:STEX:AZIN:YWZR:OPTF:6WP4:BJO7:YXQV:JSIO", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:12.122449223Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc000142770), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-2", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"KT6X:HLB7:5GNH:D266:L37O:BGAV:WIU2:5UEE:YPBY:ZGIB:SDOA:BPJE", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.507750667Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc000270070), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-1", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"G6FQ:YN5F:L6KU:WRCD:7YSX:6E4E:4NJQ:S5KA:VWGY:2R3D:ZNJ3:5RC3", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.91113988Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc000142850), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-11", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"B4FF:P2QC:JTBZ:EFWW:CSFV:ZXGH:P3KQ:7DHJ:6OTC:N4UP:6FIH:JHVB", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.676721676Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001a04d0), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-4", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"WAVS:BSTI:HWWC:QWIS:YQ7D:NXXU:GLKC:5VX6:O3LM:7EQZ:IQVB:T5TE", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.691384588Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001a05b0), NCPU:8, MemTotal:16818835456, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-6", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"CLX4:N5VS:YBZJ:OXM5:VCBV:SNEX:DX5Y:L7XF:JWRL:3CGS:IZUJ:T2SV", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.933472043Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001a0690), NCPU:8, MemTotal:16818835456, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-5", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
DEBU[0000] Docker Info found: types.Info{ID:"4X5G:Q7T2:33JQ:B7ZV:TMHP:6IZ2:GIBJ:FT5U:TQGK:WX2X:MHCN:ZZGT", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:4, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:22, OomKillDisable:true, NGoroutines:37, SystemTime:"2019-07-22T06:48:11.833279938Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"4.15.0-54-generic", OperatingSystem:"Ubuntu 18.04.2 LTS", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001a07e0), NCPU:8, MemTotal:16818843648, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"onap-k8s-9", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"18.09.5", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"894b81a4b802e4eb2a91d1ce216b8817763c29fb", Expected:"894b81a4b802e4eb2a91d1ce216b8817763c29fb"}, RuncCommit:types.Commit{ID:"425e105d5a03fabd737a126ad93d62a9eeede87f", Expected:"425e105d5a03fabd737a126ad93d62a9eeede87f"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}}
INFO[0000] [network] Deploying port listener containers
DEBU[0000] [network] Starting deployListener [rke-etcd-port-listener] on host [10.154.12.89]
DEBU[0000] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.89]
DEBU[0000] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0000] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0000] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0002] [network] Service is already up on host [10.154.12.89]
DEBU[0002] [network] Starting deployListener [rke-cp-port-listener] on host [10.154.12.89]
DEBU[0002] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.89]
DEBU[0002] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0002] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0002] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
INFO[0003] [network] Successfully started [rke-cp-port-listener] container on host [10.154.12.89]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.73]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.80]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.59]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.68]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.81]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.58]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.102]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.87]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.77]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.69]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.65]
DEBU[0003] [network] Starting deployListener [rke-worker-port-listener] on host [10.154.12.103]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.73]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.80]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.87]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.68]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.102]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.81]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.59]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.58]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.65]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.77]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.103]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0003] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.69]
DEBU[0003] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
DEBU[0003] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
DEBU[0003] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.80]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.87]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.69]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.102]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.73]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.59]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.68]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.58]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.77]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.81]
INFO[0006] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.65]
INFO[0007] [network] Successfully started [rke-worker-port-listener] container on host [10.154.12.103]
INFO[0007] [network] Port listener containers deployed successfully
INFO[0007] [network] Running control plane -> etcd port checks
DEBU[0007] [remove/rke-port-checker] Checking if container is running on host [10.154.12.89]
DEBU[0007] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.89]
DEBU[0007] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.89]
DEBU[0007] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0007] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0007] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
INFO[0008] [network] Successfully started [rke-port-checker] container on host [10.154.12.89]
DEBU[0008] [network] containerLog [] on host: 10.154.12.89
DEBU[0009] [network] Length of containerLog is [0] on host: 10.154.12.89
INFO[0009] [network] Running control plane -> worker port checks
DEBU[0009] [remove/rke-port-checker] Checking if container is running on host [10.154.12.89]
DEBU[0009] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.89]
DEBU[0009] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.89]
DEBU[0009] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0009] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0009] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
INFO[0010] [network] Successfully started [rke-port-checker] container on host [10.154.12.89]
DEBU[0010] [network] containerLog [] on host: 10.154.12.89
DEBU[0010] [network] Length of containerLog is [0] on host: 10.154.12.89
INFO[0010] [network] Running workers -> control plane port checks
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.81]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.87]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.80]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.58]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.73]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.77]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.69]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.65]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.68]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.102]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.103]
DEBU[0010] [remove/rke-port-checker] Checking if container is running on host [10.154.12.59]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.80]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.73]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.65]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.68]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.80]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.69]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.87]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.59]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.81]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.102]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.77]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.103]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.68]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.73]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.65]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0010] [remove/rke-port-checker] Container doesn't exist on host [10.154.12.58]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.69]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.87]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.81]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.77]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.103]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.59]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.102]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
DEBU[0010] [network] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.58]
DEBU[0010] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0010] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
DEBU[0010] [network] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
INFO[0012] [network] Successfully started [rke-port-checker] container on host [10.154.12.59]
DEBU[0013] [network] containerLog [] on host: 10.154.12.59
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.65]
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.80]
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.68]
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.102]
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.81]
DEBU[0013] [network] containerLog [] on host: 10.154.12.65
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.69]
DEBU[0013] [network] containerLog [] on host: 10.154.12.68
DEBU[0013] [network] containerLog [] on host: 10.154.12.80
DEBU[0013] [network] containerLog [] on host: 10.154.12.102
DEBU[0013] [network] containerLog [] on host: 10.154.12.81
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.87]
DEBU[0013] [network] containerLog [] on host: 10.154.12.69
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.77]
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.58]
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.103]
DEBU[0013] [network] containerLog [] on host: 10.154.12.103
INFO[0013] [network] Successfully started [rke-port-checker] container on host [10.154.12.73]
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.69
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.59
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.80
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.102
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.68
DEBU[0014] [network] containerLog [] on host: 10.154.12.58
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.103
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.58
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.65
DEBU[0014] [network] containerLog [] on host: 10.154.12.73
DEBU[0014] [network] containerLog [] on host: 10.154.12.77
DEBU[0014] [network] containerLog [] on host: 10.154.12.87
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.81
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.87
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.77
DEBU[0014] [network] Length of containerLog is [0] on host: 10.154.12.73
INFO[0014] [network] Checking KubeAPI port Control Plane hosts
DEBU[0014] [network] Checking KubeAPI port [6443] on host: 10.154.12.89
INFO[0014] [network] Removing port listener containers
DEBU[0014] [remove/rke-etcd-port-listener] Checking if container is running on host [10.154.12.89]
DEBU[0014] [remove/rke-etcd-port-listener] Removing container on host [10.154.12.89]
INFO[0014] [remove/rke-etcd-port-listener] Successfully removed container on host [10.154.12.89]
DEBU[0014] [remove/rke-cp-port-listener] Checking if container is running on host [10.154.12.89]
DEBU[0014] [remove/rke-cp-port-listener] Removing container on host [10.154.12.89]
INFO[0015] [remove/rke-cp-port-listener] Successfully removed container on host [10.154.12.89]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.87]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.73]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.81]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.80]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.58]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.59]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.68]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.102]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.77]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.65]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.69]
DEBU[0015] [remove/rke-worker-port-listener] Checking if container is running on host [10.154.12.103]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.65]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.68]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.81]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.58]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.87]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.59]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.69]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.102]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.73]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.103]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.80]
DEBU[0015] [remove/rke-worker-port-listener] Removing container on host [10.154.12.77]
INFO[0016] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.103]
INFO[0016] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.87]
INFO[0016] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.102]
INFO[0016] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.68]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.77]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.81]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.59]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.69]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.80]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.73]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.65]
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [10.154.12.58]
INFO[0017] [network] Port listener containers removed successfully
INFO[0017] [certificates] Deploying kubernetes certificates to Cluster nodes
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.68]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.59]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.77]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.69]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.103]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.73]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.80]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.89]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.58]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.65]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.102]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.81]
DEBU[0017] Checking if container [cert-deployer] is running on host [10.154.12.87]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.77]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.103]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.73]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.68]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.58]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.65]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.80]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.89]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.59]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.69]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.81]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.87]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.68]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.58]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.103]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.73]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.87]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.65]
DEBU[0017] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.102]
DEBU[0017] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.69]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.81]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.77]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.59]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.80]
DEBU[0017] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
DEBU[0017] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.102]
DEBU[0019] [certificates] Successfully started Certificate deployer container: 3442dccda2fdc06bea195285f4e82cc604705b09e6476f1527fa3244f3fe450a
DEBU[0019] Checking if container [cert-deployer] is running on host [10.154.12.58]
DEBU[0019] [certificates] Successfully started Certificate deployer container: 25d97dde793b0a112073dc44c4da6cda1d3d3f55edfb1a68e8c603c2b76aa17b
DEBU[0019] Checking if container [cert-deployer] is running on host [10.154.12.102]
DEBU[0019] [certificates] Successfully started Certificate deployer container: 9063483340ff397eff2006a432826e22fdc5ae0cd1e96d8a702e960e6415d602
DEBU[0019] Checking if container [cert-deployer] is running on host [10.154.12.87]
DEBU[0019] [certificates] Successfully started Certificate deployer container: bebfb4a385024a73489a0c8498421d757e87f954ccf509451f1aea24001108ed
DEBU[0019] Checking if container [cert-deployer] is running on host [10.154.12.73]
DEBU[0019] [certificates] Successfully started Certificate deployer container: fd348ace30270ecd4bb6a8a72739e48d91fdf8a692979eb7a85baf0b0e8db8f3
DEBU[0019] Checking if container [cert-deployer] is running on host [10.154.12.89]
DEBU[0019] [certificates] Successfully started Certificate deployer container: e46353f31475ed015c13369517bdc98d0cc09657aaab6646888a1ca39db65240
DEBU[0019] Checking if container [cert-deployer] is running on host [10.154.12.68]
DEBU[0019] [certificates] Successfully started Certificate deployer container: 151a12fc95a8513fbbb630adb177aff3fe5db44eb2a7fa411c067a86babdae9f
DEBU[0019] Checking if container [cert-deployer] is running on host [10.154.12.77]
DEBU[0019] [certificates] Successfully started Certificate deployer container: 2a4cd93c50562cb90d5d851f9105188c78ca7ee9da3093efee75885808a5ec05
DEBU[0019] Checking if container [cert-deployer] is running on host [10.154.12.103]
DEBU[0020] [certificates] Successfully started Certificate deployer container: 034e62270a8d5e8aedcf92dc701e00239e639f9626ac480cec6c1c4fc99a0414
DEBU[0020] Checking if container [cert-deployer] is running on host [10.154.12.80]
DEBU[0020] [certificates] Successfully started Certificate deployer container: 955bdf5ff0861517fbd40c157ba1e032d0f61f4b26bf3697c2c9888225483941
DEBU[0020] Checking if container [cert-deployer] is running on host [10.154.12.81]
DEBU[0020] [certificates] Successfully started Certificate deployer container: 937883a4f6357aa55251620ac78930924ad103e4a7c3e6d89460c7c50d4eedda
DEBU[0020] Checking if container [cert-deployer] is running on host [10.154.12.65]
DEBU[0020] [certificates] Successfully started Certificate deployer container: 74da8262c0d1f1ed9b9f700b22117bfca2a91ad0e0dc36200c75a0c6c78a6192
DEBU[0020] Checking if container [cert-deployer] is running on host [10.154.12.69]
DEBU[0020] [certificates] Successfully started Certificate deployer container: 31a64e6377f98c62bd4bbb34f5f84a7d6d59911593bc967814c33cbe06c2369d
DEBU[0020] Checking if container [cert-deployer] is running on host [10.154.12.59]
DEBU[0024] Checking if container [cert-deployer] is running on host [10.154.12.58]
DEBU[0024] Checking if container [cert-deployer] is running on host [10.154.12.102]
DEBU[0024] Checking if container [cert-deployer] is running on host [10.154.12.87]
DEBU[0024] Checking if container [cert-deployer] is running on host [10.154.12.73]
DEBU[0024] Checking if container [cert-deployer] is running on host [10.154.12.68]
DEBU[0024] Checking if container [cert-deployer] is running on host [10.154.12.89]
DEBU[0024] Checking if container [cert-deployer] is running on host [10.154.12.77]
DEBU[0024] Checking if container [cert-deployer] is running on host [10.154.12.103]
DEBU[0025] Checking if container [cert-deployer] is running on host [10.154.12.80]
DEBU[0025] Checking if container [cert-deployer] is running on host [10.154.12.81]
DEBU[0025] Checking if container [cert-deployer] is running on host [10.154.12.65]
DEBU[0025] Checking if container [cert-deployer] is running on host [10.154.12.69]
DEBU[0025] Checking if container [cert-deployer] is running on host [10.154.12.59]
INFO[0025] [reconcile] Rebuilding and updating local kube config
DEBU[0025] Deploying admin Kubeconfig locally: apiVersion: v1
kind: Config
clusters:
- cluster:
api-version: v1
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN3akNDQWFxZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFTTVJBd0RnWURWUVFERXdkcmRXSmwKTFdOaE1CNFhEVEU1TURjeU1qQTVNekkxTkZvWERUSTVNRGN4T1RBNU16STFORm93RWpFUU1BNEdBMVVFQXhNSAphM1ZpWlMxallUQ0NBU0l3RFFZSktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQUxzR2czdk1BMFJNCm5wdjFFamY0UFg3WXBBbVIwREdlanhTNGNZa3BZYWYzRVBwSFhiSWJPRm1WUzVldGcxYm4zeWRhV1U2ZzQrS1EKMDllQ1hEakVUbjBKVjFtSlZ1TFhoaG0rUWtqOWVpRGI2UU1pcWwxbE81OWFKV0xPTHdrdlJDblpISTNuY0lEVwpVY0t0OVJsN0F3ZDNUQWZSSFFTTHVFdWVkSmlEYnR4alFRUzNpdFZOSjNLaFZkZWFoTjl6Qy9NUWlxUmQ4OGpsCjZFcEZjSnp3eERHR1VFc0N3cUV0V3NGVXgrNS9iam1xRXBUbUlIbWlVWkFteUJhYjFSMERua3JDYVdXeFpHWmcKUm9kQlhyU0RpVnlBOW1MWEJiWDdNdSt0TURIa1hSY3FmclRNN0FMWkIvYThWNWJQQkcrNEFtb0pRNE9lRStsTQpVbTFIRVRuOUVrVUNBd0VBQWFNak1DRXdEZ1lEVlIwUEFRSC9CQVFEQWdLa01BOEdBMVVkRXdFQi93UUZNQU1CCkFmOHdEUVlKS29aSWh2Y05BUUVMQlFBRGdnRUJBRjMvWWdnRlFaR00weTdrTU5Cb1l1ODBCWmtyTjMyUU9TbzcKQVNURzFYZ2FwQ0o2OG1tbzVVd1NZbERkQ09jRUErWXBBOHBQVExOd1NuR1ZCRDZPcVVoeWZHMXU0THJ6N09qVApvcnpFd05HZXhqUlFCQ2ZWNDVtdnpIeEsrQkxwbHBMeVpKMjd2ZWZGN0FFSHFMK1RnNDNqWlVkWDZuK3JFb1hvCm0xTUVzYXpqd0VGS0VkZStvcnFtM1d5MFgwUHNkSGo0U0Z0SUVNSHl4WlR4NytpMjJoSE1IOUxDQVpVTStGZEEKdjl0MnNjUWxqb1loQWRseHJSbkt0dDZDK2hldGtTN2JQVlIwRCsxL3NxMUwzTFdZVTB3S2tDWitVdUwzeXJhcAoxSlAyRVMxb3kyYlVxQi8xcjlRMDNmU1JuOStzblhoMmkzdmtza0kybmlIRThpdnBEYTA9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
server: "https://10.154.12.89:6443"
name: "onap"
contexts:
- context:
cluster: "onap"
user: "kube-admin-onap"
name: "onap"
current-context: "onap"
users:
- name: "kube-admin-onap"
user:
client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM2VENDQWRHZ0F3SUJBZ0lJSlpva21qY2lLamN3RFFZSktvWklodmNOQVFFTEJRQXdFakVRTUE0R0ExVUUKQXhNSGEzVmlaUzFqWVRBZUZ3MHhPVEEzTWpJd09UTXlOVFJhRncweU9UQTNNVGt3T1RRd05UaGFNQzR4RnpBVgpCZ05WQkFvVERuTjVjM1JsYlRwdFlYTjBaWEp6TVJNd0VRWURWUVFERXdwcmRXSmxMV0ZrYldsdU1JSUJJakFOCkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQW9UQW1yNmRlbnJHVmh0eU4yOE5XMjZOR2xLWk0KN1RCUnNGSmJ6WEtzTHdzMVFMcTdwMmFNaWpoRUttOVdYZ3JyeUR1dTUrS2ZqYWtDT3pmQ3J2WXRZUXNGMmxlWAo2U241eHROeHNWd0NKK2h4cm5mS24rd1ZHTWZwZFhZZjdhTjF3UW1pemVYekF6VU9UaFJOZW9uc3A2Yndma0JrCnpZRHRFTW5SWjNlZExwcG9OQVZtTzA3S1JoODlBZitNd3NBT201Z2xIeXFDTmx5VnprUFEzclJ6U25PZzU5WFgKUjBDNld4eDZMWUlVdUxUREVoN1VFZVE5aXlJdGV4MjRRaVFZVmRXUEhZWEVWZ1VSL1RHMUV4bFlHa2FValIrWQovaTY2SVdmOFNGN0tVeTV2QkZGVHFzbHRIdWtVNjEydEc1emU0QUxsbXpoTWJJblh1SGxTdVBuUUNRSURBUUFCCm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0l3RFFZSktvWkkKaHZjTkFRRUxCUUFEZ2dFQkFBMWcrVDk4a0d5Z0Q3YVE3N3dRbnJXRkp5U2lQQ21UUmF4K3c1L2RQUXBkUm9pSwplUEc0MjRXY2hoRWZwVi9nSHlHM3NiOU5MU1BkZS9zYTF2M2RsS1hNWGNvTHFlUTI2b09FL2VXY1hiL1FVck5JCk1JQW9BYWUwL041T0hmSHhiVFI2Q084b21mSjdTWldVbXQ5c1dYZDlyb3J3dHhuN0tVOEYzam9uK0hGT3dCMVoKbTd6VUg2L1g4OTJUMFFaZTZPWHRzdEgyMEIzY1kzR2hvaVdZS08rT29JWkN1MXBGVVFvR2YzWGtpMDYwdHp4bQpoRXNwbU9CSGg0RGRqNm9YTnlobFlQSFlqeWFZMFF4akRkNjhpVEx2YUdCMm04SzZvRE9HVm8reVNoT3ZRa2wvCkc3R1BIUFlkclNyWi93MFJWTW1GOEpNSEoydU85TEphTFgzbTlSWT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBb1RBbXI2ZGVuckdWaHR5TjI4TlcyNk5HbEtaTTdUQlJzRkpielhLc0x3czFRTHE3CnAyYU1pamhFS205V1hncnJ5RHV1NStLZmpha0NPemZDcnZZdFlRc0YybGVYNlNuNXh0TnhzVndDSitoeHJuZksKbit3VkdNZnBkWFlmN2FOMXdRbWl6ZVh6QXpVT1RoUk5lb25zcDZid2ZrQmt6WUR0RU1uUlozZWRMcHBvTkFWbQpPMDdLUmg4OUFmK013c0FPbTVnbEh5cUNObHlWemtQUTNyUnpTbk9nNTlYWFIwQzZXeHg2TFlJVXVMVERFaDdVCkVlUTlpeUl0ZXgyNFFpUVlWZFdQSFlYRVZnVVIvVEcxRXhsWUdrYVVqUitZL2k2NklXZjhTRjdLVXk1dkJGRlQKcXNsdEh1a1U2MTJ0RzV6ZTRBTGxtemhNYkluWHVIbFN1UG5RQ1FJREFRQUJBb0lCQUdQUmx0T0dlWEViRmpjcgpGR3A0d0t4Sm82YmdpWmtGS0VGaDcxbitYcVRQY3czLzFiL29JTFpqKzhxdm9qT2czYnpsUVA1MitFblkxUG5wCklSWUtlN1d5QnREZEY4VWI2b04zWTl2cTE2YnhLcDFacjlCemVla2J1L2F5elRqejFqTlNEVkVteXVROGtmeEcKVkdqRFN4dlRwUEQ2WGJKcDJlVmdwZDVHVTcrTWdiczJWUW5kenpCUDJpakpsZjduYUNXcjRSeTFIbHFNVGRYaApxUkhSbDJyRjcvaE81YmY1VnYxSHZNL1AyWWN5OTRxNHYyS3FxdmFHRW9kOFhraUlZckcxd0NyT2N2V0F1eW4yCkNjbFdoOFNCM0JBcmU0YmtwNEI1VzNVMGxiekkxSzZUNmJNZXQ1dFI3K1FyTmg3cGRTY29tQ09ldFljU1FZeXAKUjJtaDJKRUNnWUVBei9pY1o2bk82T2h6YmdqZ2NhSzZRUm93US9FWlRzaHRJQnh5OUtBaDQ3ZHlDTnludzRXWAppbGVkQ2tBZDJNREFsYVBZdTg4RFZXOEZ2WU5XK1FOdElaMEJrc0xGQ0pwdWFCdHd0V1pRVkF5NlMwZFFYYnJGCmNDYlVremszVUFKTFBNOGs4UVpab3FmWDFRL2RwcHI4VnB3cW1XSkwyL2RyWGRPWmlwc2NlajBDZ1lFQXhtbXkKTE9LSlFOaDFINmJKdnBzeXBva2xMVERoQlp5dkxOQk43S1E4bEJOZmZJUGQ4eDFjMGVLeFkwQm5WZVNpNlBtSApOWUJZNDRLZjJLeEpMaDN0MmpROGd5S05vU0lmZVhjcnE3RHlRdlFjZHhSU1h4dWU4TEVhb3VkdGdCK0J3RVpKCjhxK1dEaTlUYjNGQ0RtWWpId2RwZG1BdGU5NmVGWlVaNDJLMzViMENnWUVBbGltVUJPVi9HSHgrbS95YzBuQUUKdSswRTNJbkJGRGx6VWJRNDdKaVMzN0VkRFhrRlNRKzBrNWNmTkwrR3pYS2ZtMnhxbVhDbm41QkNQTXdRaUV3SgpDRFF0WnFKclVnTHZwUXZySFk5NmdMV1A3YnJZTkZVZktoTmZlYzNlT2s3dHRFSjlFZXJiM2E0REVUd2JkcWlCCmVxZlh1bGhYSGNSUUtiY1hMaFZCUXFFQ2dZRUFuL0NCMVZiRFowZFZtcThiUDQvaGs0ZElOWmFLUGJqaEtORzkKanYyUmE1ZjJSK3dRbFRKZEtUK0EyTTRJdnA5d2h6Ym1QRVY2NGVMaE1Zc243b25udXhhU2xPN2Nsa0JvZU1TTQpxbmtYd01zQXl5alJnYU5GU3lhb254ckpGUmZRd3dxbC80QjFDSGd6M01qQ2c5TG5pdEt4dTkrcHFvbDRpMU5HCnJCcGV2VTBDZ1lBcWx1cWUwcHZTd0haR0U1NW9NbXYzZWlPVGs1TlFrZnVJaUJJcEUwLzQvMW5JNjdWWkI0UWkKWjQ0WlBuNzdtei9jUFZyeisrMFFub1pOQnoxS1hsVWYybFJPZXJDYW01N0plM0lYOVJrU3RqbmJaT3V1SXZ4Ngo4S2ZRQkNxM1ZIY1VwVEk3a2xFaFpxNE1VVzJBOGwwRkorMjMxU01XYlBQNy8zY2VFRlU4Ymc9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
INFO[0025] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml]
DEBU[0025] [version] Using ./kube_config_cluster.yml to connect to Kubernetes cluster..
DEBU[0025] [version] Getting Kubernetes server version..
INFO[0025] [certificates] Successfully deployed kubernetes certificates to Cluster nodes
INFO[0025] [reconcile] Reconciling cluster state
INFO[0025] [reconcile] This is newly generated cluster
INFO[0025] Pre-pulling kubernetes images
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.102]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.87]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.102]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.87]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.89]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.89]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.80]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.80]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.73]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.68]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.73]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.68]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.77]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.77]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.65]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.65]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.81]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.69]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.58]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.69]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.58]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.81]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.103]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.103]
DEBU[0025] [pre-deploy] Checking image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] on host [10.154.12.59]
DEBU[0025] Checking if image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.59]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.89]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.89]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.68]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.68]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.102]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.102]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.80]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.80]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.87]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.87]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.81]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.81]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.69]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.69]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.73]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.73]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.77]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.77]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.58]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.58]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.59]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.59]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.65]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.65]
DEBU[0025] Image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.103]
DEBU[0025] [pre-deploy] No pull necessary, image [nexus3.onap.org:10001/rancher/hyperkube:v1.13.4-rancher1] exists on host [10.154.12.103]
INFO[0025] Kubernetes images pulled successfully
DEBU[0025] Extracted version [v0.1.16] from image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16]
DEBU[0025] Extracted version [v0.1.16] from image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16]
DEBU[0025] Extracted version [v0.1.16] from image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16]
DEBU[0025] Extracted version [v0.1.16] from image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16]
DEBU[0025] Extracted version [v0.1.16] from image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16]
INFO[0025] [etcd] Building up etcd plane..
DEBU[0025] [etcd] Container [etcd] is already running on host [10.154.12.89]
DEBU[0025] [etcd] Checking if container [etcd] is eligible for upgrade on host [10.154.12.89]
DEBU[0025] [etcd] Container [etcd] is not eligible for upgrade on host [10.154.12.89]
INFO[0025] [etcd] Saving snapshot [etcd-rolling-snapshots] on host [10.154.12.89]
DEBU[0025] [remove/etcd-rolling-snapshots] Checking if container is running on host [10.154.12.89]
DEBU[0025] [remove/etcd-rolling-snapshots] Removing container on host [10.154.12.89]
INFO[0026] [remove/etcd-rolling-snapshots] Successfully removed container on host [10.154.12.89]
DEBU[0026] [etcd] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.89]
DEBU[0026] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0026] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0026] [etcd] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
INFO[0027] [etcd] Successfully started [etcd-rolling-snapshots] container on host [10.154.12.89]
DEBU[0032] [certificates] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.89]
DEBU[0032] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0032] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0032] [certificates] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
INFO[0033] [certificates] Successfully started [rke-bundle-cert] container on host [10.154.12.89]
INFO[0034] [certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [10.154.12.89]
DEBU[0034] [etcd] Creating log link for Container [etcd] on host [10.154.12.89]
DEBU[0034] [remove/rke-log-linker] Checking if container is running on host [10.154.12.89]
DEBU[0034] [remove/rke-log-linker] Container doesn't exist on host [10.154.12.89]
DEBU[0034] [etcd] Checking image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] on host [10.154.12.89]
DEBU[0034] Checking if image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0034] Image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
DEBU[0034] [etcd] No pull necessary, image [nexus3.onap.org:10001/rancher/rke-tools:v0.1.16] exists on host [10.154.12.89]
INFO[0034] [etcd] Successfully started [rke-log-linker] container on host [10.154.12.89]
DEBU[0034] [remove/rke-log-linker] Checking if container is running on host [10.154.12.89]
DEBU[0034] [remove/rke-log-linker] Removing container on host [10.154.12.89]
INFO[0035] [remove/rke-log-linker] Successfully removed container on host [10.154.12.89]
DEBU[0035] [etcd] Successfully created log link for Container [etcd] on host [10.154.12.89]
INFO[0035] [etcd] Successfully started etcd plane.. Checking etcd cluster health
DEBU[0035] [etcd] Check etcd cluster health
DEBU[0036] Failed to get /health for host [10.154.12.89]: Get https://172.16.100.40:2379/health: remote error: tls: bad certificate
DEBU[0041] Failed to get /health for host [10.154.12.89]: Get https://172.16.100.40:2379/health: remote error: tls: bad certificate
DEBU[0047] Failed to get /health for host [10.154.12.89]: Get https://172.16.100.40:2379/health: remote error: tls: bad certificate
FATA[0052] [etcd] Failed to bring up Etcd Plane: [etcd] Etcd Cluster is not healthy
Also, this is the full log for etcd container in the control node:
root@onap-control-1:~# docker logs 78bfbe568c0a
2019-07-22 06:45:41.360303 W | pkg/flags: unrecognized environment variable ETCD_UNSUPPORTED_ARCH=x86_64
2019-07-22 06:45:41.360370 I | etcdmain: etcd Version: 3.2.24
2019-07-22 06:45:41.360374 I | etcdmain: Git SHA: 420a45226
2019-07-22 06:45:41.360377 I | etcdmain: Go Version: go1.8.7
2019-07-22 06:45:41.360380 I | etcdmain: Go OS/Arch: linux/amd64
2019-07-22 06:45:41.360384 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2019-07-22 06:45:41.360429 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-172-16-100-40.pem, key = /etc/kubernetes/ssl/kube-etcd-172-16-100-40-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2019-07-22 06:45:41.361036 I | embed: listening for peers on https://172.16.100.40:2380
2019-07-22 06:45:41.361084 I | embed: listening for client requests on 172.16.100.40:2379
2019-07-22 06:45:41.504447 I | etcdserver: name = etcd-onap-control-1
2019-07-22 06:45:41.504465 I | etcdserver: data dir = /var/lib/rancher/etcd/
2019-07-22 06:45:41.504474 I | etcdserver: member dir = /var/lib/rancher/etcd/member
2019-07-22 06:45:41.504479 I | etcdserver: heartbeat = 500ms
2019-07-22 06:45:41.504484 I | etcdserver: election = 5000ms
2019-07-22 06:45:41.504489 I | etcdserver: snapshot count = 100000
2019-07-22 06:45:41.504499 I | etcdserver: advertise client URLs = https://172.16.100.40:2379,https://172.16.100.40:4001
2019-07-22 06:45:41.504508 I | etcdserver: initial advertise peer URLs = https://172.16.100.40:2380
2019-07-22 06:45:41.504516 I | etcdserver: initial cluster = etcd-onap-control-1=https://172.16.100.40:2380
2019-07-22 06:45:41.599902 I | etcdserver: starting member 9019c714a7a7fe3b in cluster ad1b89b5edcdbfd3
2019-07-22 06:45:41.599933 I | raft: 9019c714a7a7fe3b became follower at term 0
2019-07-22 06:45:41.599944 I | raft: newRaft 9019c714a7a7fe3b [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]
2019-07-22 06:45:41.599950 I | raft: 9019c714a7a7fe3b became follower at term 1
2019-07-22 06:45:41.707948 W | auth: simple token is not cryptographically signed
2019-07-22 06:45:41.714571 I | etcdserver: starting server... [version: 3.2.24, cluster version: to_be_decided]
2019-07-22 06:45:41.714971 I | etcdserver: 9019c714a7a7fe3b as single-node; fast-forwarding 9 ticks (election ticks 10)
2019-07-22 06:45:41.715256 I | etcdserver/membership: added member 9019c714a7a7fe3b [https://172.16.100.40:2380] to cluster ad1b89b5edcdbfd3
2019-07-22 06:45:41.715457 I | embed: ClientTLS: cert = /etc/kubernetes/ssl/kube-etcd-172-16-100-40.pem, key = /etc/kubernetes/ssl/kube-etcd-172-16-100-40-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true
2019-07-22 06:45:44.600162 I | raft: 9019c714a7a7fe3b is starting a new election at term 1
2019-07-22 06:45:44.600221 I | raft: 9019c714a7a7fe3b became candidate at term 2
2019-07-22 06:45:44.600241 I | raft: 9019c714a7a7fe3b received MsgVoteResp from 9019c714a7a7fe3b at term 2
2019-07-22 06:45:44.600258 I | raft: 9019c714a7a7fe3b became leader at term 2
2019-07-22 06:45:44.600265 I | raft: raft.node: 9019c714a7a7fe3b elected leader 9019c714a7a7fe3b at term 2
2019-07-22 06:45:44.600606 I | etcdserver: published {Name:etcd-onap-control-1 ClientURLs:[https://172.16.100.40:2379 https://172.16.100.40:4001]} to cluster ad1b89b5edcdbfd3
2019-07-22 06:45:44.600630 I | embed: ready to serve client requests
2019-07-22 06:45:44.600667 I | etcdserver: setting up the initial cluster version to 3.2
2019-07-22 06:45:44.600921 I | embed: serving client requests on 172.16.100.40:2379
2019-07-22 06:45:44.602558 N | etcdserver/membership: set the initial cluster version to 3.2
2019-07-22 06:45:44.602626 I | etcdserver/api: enabled capabilities for version 3.2
WARNING: 2019/07/22 06:45:44 Failed to dial 172.16.100.40:2379: connection error: desc = "transport: authentication handshake failed: remote error: tls: bad certificate"; please retry.
I have the similar trouble when upgrading from (rke 0.2.4/k8s 1.13.5) to (rke 0.2.6/k8s 1.14.3). Environment is Ubuntu on OpenStack. The clean install works well though.
Strange thing I noticed though, even though I have all three of .yml, *.rkestate and kube_config_.yml files in the work directory, I still see this message in the logs:
...
DEBU[0000] [state] previous state found, this is not a legacy cluster
INFO[0000] [certificates] Generating admin certificates and kubeconfig
...
INFO[0016] [network] Port listener containers removed successfully
INFO[0016] [certificates] Deploying kubernetes certificates to Cluster nodes
...
INFO[0023] [reconcile] Rebuilding and updating local kube config
...
INFO[0032] [reconcile] Reconciling cluster state
INFO[0032] [reconcile] This is newly generated cluster
...
DEBU[0067] [etcd] Successfully created log link for Container [etcd] on host [rnc02.infra.example.com]
INFO[0067] [etcd] Successfully started etcd plane.. Checking etcd cluster health
DEBU[0067] [etcd] Check etcd cluster health
DEBU[0067] Failed to get /health for host [rnc00.infra.example.com]: Get https://rnc00.infra.example.com:2379/health: remote error: tls: bad certificate
...
FATA[0113] [etcd] Failed to bring up Etcd Plane: [etcd] Etcd Cluster is not healthy
After that:
- the cluster k8s has not upgraded
- I am left with the kube_config*.yml file that has totally different keys and cert from the original one, and it won't work with the cluster
- The original kube_config*.yml still works fine with the cluster.
- the "server" parameter in the new kube_config.yml points to a different host compared to original kube_config.yml
P.S. If full log is needed, I can sanitize and attach it. Also, I am free for experimentation on request.
verikca
on 24 Jul 2019
Hi, we are experiencing same issue with rke 0.2.4 and k8s v1.14.3. on completely wiped VMs with rke 0.1.18 it works just fine. Please @galal-hussein let us know where to look as I'm 100% positive that there are no leftover certificates as I have followed https://rancher.com/docs/rancher/v2.x/en/cluster-admin/cleaning-cluster-nodes/.
Thanks
p.s. @sarpkoksal can you please edit your comments and move outputs to https://gist.github.com as your long post makes hard to navigate this issue. Thanks
elmariofredo
on 26 Jul 2019
Can you share the output of this script (which collects certificate info of the node):
#!/bin/sh
if [ -d /opt/rke/etc/kubernetes ]; then
K8S_DIR="/opt/rke/etc/kubernetes"
else
K8S_DIR="/etc/kubernetes"
fi
for cert in `find "${K8S_DIR}/ssl" -type f -name *.pem | grep -v "\-key\.pem$" | grep -v kube-admin | sort`; do
md5sum $cert
done
if [ -d "${K8S_DIR}/.tmp" ]; then
for cert in `find "${K8S_DIR}/.tmp" -type f -name *.pem | grep -v "\-key\.pem$" | grep -v kube-admin | sort`; do md5sum $cert; done
fi
if command -v openssl >/dev/null 2>&1; then
if [ -f "${K8S_DIR}/ssl/kube-ca.pem" ]; then
for cert in `find "${K8S_DIR}/ssl" -type f -name *.pem | grep -v "\-key\.pem$" | grep -v kube-admin | sort`; do
if [ "$(basename $cert)" = "kube-apiserver-proxy-client.pem" ] || [ "$(basename $cert)" = "kube-apiserver-requestheader-ca.pem" ] || [ "$(basename $cert)" = "kube-ca.pem" ]; then
continue
fi
SSL_CERT_DIR=/dummy
SSL_CERT_FILE=/dummy
openssl verify -CAfile "${K8S_DIR}/ssl/kube-ca.pem" $cert
done
fi
fi
find "${K8S_DIR}/ssl" -type f -name *.pem -exec ls -la {} \; | sort
@elmariofredo If you can share debug output from first run on a clean node, that would be helpful too.
superseb
on 26 Jul 2019
@superseb output was empty, but eventually we have find out that it's failing only from our CI and works from our local machines and problem was wrongly setup ntp server.
Seems that rke generates certs on machine where is executed and not on machine where is it deploying k8s, so when machine where rke up is executed have different time than target machines we end up with this error.
Would be great idea to either check that or at least add extra note to error output to suggest ntp update.
elmariofredo
on 8 Aug 2019
That is covered in https://github.com/rancher/rke/issues/1123
If others can supply the requested outputs, we can continue investigating.
superseb
on 8 Aug 2019
👍1
@superseb FYI, there is another issue with the same symptoms as I described a few messages above which I worked around as following.
I was upgrading a cluster running in OpenStack with RKE 0.2.6 (and running Rancher 2.2.6). The cluster was originally created with RKE 0.2.3 (and running Rancher 2.2.3). At one point when upgrading Rancher on this cluster we did bump into the issue described in https://github.com/rancher/rancher/issues/20699
So right now, when doing the k8s upgrade for this cluster we did face the same issue again, only on RKE level. What was happening, the cluster.rkestate file we had was from RKE 0.2.3 and it had the following lines inside (two occurrences):
"monitor-delay": 0,
"monitor-timeout": 0,
These seem to be illegal values for the new RKE so RKE was simply ignoring my cluster.rkestate file replacing it with a new one and generating new certificates. Unfortunately the new certificates were not able to talk to etcd in the cluster causing the exact same symptoms the original poster was having.
The fix for me was to change those values in cluster.rkestate file (make sure you use the original one) to:
"monitor-delay": "",
"monitor-timeout": "",
(two occurrences)
Then upgrade succeeded.
P.S. I also tried to restore the backup instead of upgrading and faced the same issue as I described in the comment in https://github.com/rancher/rke/issues/1362
verikca
on 8 Aug 2019
I have the similar trouble when added one node in config.yml (2 nodes update to 3 nodes)
$rke up --update-only --config=config.yml
$rke --version
rke version v0.2.8
certificate output: ( this output from superseb's script )
K8S_DIR=/etc/kubernetes
bb92a4d67f1d65a13fe2ed8e34e7f18a /etc/kubernetes/ssl/kube-apiserver.pem
1f337c5f659de378003594c64b64fb84 /etc/kubernetes/ssl/kube-apiserver-proxy-client.pem
aaa5f9c1f2a8f0c526de9fe0912224cc /etc/kubernetes/ssl/kube-apiserver-requestheader-ca.pem
58cc346457d71d2b57bf089ea698ef24 /etc/kubernetes/ssl/kube-ca.pem
ebb9eea009dde2ed058914dc492f645d /etc/kubernetes/ssl/kube-controller-manager.pem
3cad2cf2965610d44d79761d178bd2d9 /etc/kubernetes/ssl/kube-etcd-192-168-9-63.pem
4026453a92144a7439d0d85012e8cd28 /etc/kubernetes/ssl/kube-node.pem
10c86c1e2abed7e160db0185986cd1fc /etc/kubernetes/ssl/kube-proxy.pem
17cac50effe508db2e2ccf0432e2efbc /etc/kubernetes/ssl/kube-scheduler.pem
7352ef719dc7b773ff67793b0f51b079 /etc/kubernetes/ssl/kube-service-account-token.pem
/etc/kubernetes/ssl/kube-apiserver.pem: OK
/etc/kubernetes/ssl/kube-controller-manager.pem: OK
/etc/kubernetes/ssl/kube-etcd-192-168-9-63.pem: OK
/etc/kubernetes/ssl/kube-node.pem: OK
/etc/kubernetes/ssl/kube-proxy.pem: OK
/etc/kubernetes/ssl/kube-scheduler.pem: OK
/etc/kubernetes/ssl/kube-service-account-token.pem: OK
-rw------- 1 root root 1017 Aug 29 17:26 /etc/kubernetes/ssl/kube-ca.pem
-rw------- 1 root root 1046 Aug 29 17:26 /etc/kubernetes/ssl/kube-proxy.pem
-rw------- 1 root root 1050 Aug 29 17:26 /etc/kubernetes/ssl/kube-scheduler.pem
-rw------- 1 root root 1058 Aug 29 17:26 /etc/kubernetes/ssl/kube-service-account-token.pem
-rw------- 1 root root 1062 Aug 29 17:26 /etc/kubernetes/ssl/kube-controller-manager.pem
-rw------- 1 root root 1070 Aug 29 17:26 /etc/kubernetes/ssl/kube-node.pem
-rw------- 1 root root 1082 Aug 29 17:26 /etc/kubernetes/ssl/kube-apiserver-requestheader-ca.pem
-rw------- 1 root root 1107 Aug 29 17:26 /etc/kubernetes/ssl/kube-apiserver-proxy-client.pem
-rw------- 1 root root 1265 Aug 29 17:26 /etc/kubernetes/ssl/kube-etcd-192-168-9-63.pem
-rw------- 1 root root 1273 Aug 29 17:26 /etc/kubernetes/ssl/kube-apiserver.pem
-rw------- 1 root root 1675 Aug 29 17:26 /etc/kubernetes/ssl/kube-apiserver-key.pem
-rw------- 1 root root 1675 Aug 29 17:26 /etc/kubernetes/ssl/kube-apiserver-requestheader-ca-key.pem
-rw------- 1 root root 1675 Aug 29 17:26 /etc/kubernetes/ssl/kube-controller-manager-key.pem
-rw------- 1 root root 1675 Aug 29 17:26 /etc/kubernetes/ssl/kube-scheduler-key.pem
-rw------- 1 root root 1675 Aug 29 17:26 /etc/kubernetes/ssl/kube-service-account-token-key.pem
-rw------- 1 root root 1679 Aug 29 17:26 /etc/kubernetes/ssl/kube-apiserver-proxy-client-key.pem
-rw------- 1 root root 1679 Aug 29 17:26 /etc/kubernetes/ssl/kube-ca-key.pem
-rw------- 1 root root 1679 Aug 29 17:26 /etc/kubernetes/ssl/kube-etcd-192-168-9-63-key.pem
-rw------- 1 root root 1679 Aug 29 17:26 /etc/kubernetes/ssl/kube-node-key.pem
-rw------- 1 root root 1679 Aug 29 17:26 /etc/kubernetes/ssl/kube-proxy-key.pem
*rke log:
INFO[0015] [etcd] Successfully started [rke-log-linker] container on host [192.168.9.63]
DEBU[0015] [remove/rke-log-linker] Checking if container is running on host [192.168.9.63]
DEBU[0015] [remove/rke-log-linker] Removing container on host [192.168.9.63]
INFO[0015] [remove/rke-log-linker] Successfully removed container on host [192.168.9.63]
DEBU[0015] [etcd] Successfully created log link for Container [etcd] on host [192.168.9.63]
INFO[0015] [etcd] Successfully started etcd plane.. Checking etcd cluster health
DEBU[0015] [etcd] Check etcd cluster health
DEBU[0015] Failed to get /health for host [192.168.9.63]: Get https://192.168.9.63:2379/health: remote error: tls: bad certificate
DEBU[0020] Failed to get /health for host [192.168.9.63]: Get https://192.168.9.63:2379/health: remote error: tls: bad certificate
DEBU[0025] Failed to get /health for host [192.168.9.63]: Get https://192.168.9.63:2379/health: remote error: tls: bad certificate
FATA[0030] [etcd] Failed to bring up Etcd Plane: [etcd] Etcd Cluster is not healthy
**docker logs from etcd
2019-08-29 09:26:33.875644 W | pkg/flags: unrecognized environment variable ETCD_UNSUPPORTED_ARCH=x86_64
2019-08-29 09:26:33.875760 I | etcdmain: etcd Version: 3.3.10
2019-08-29 09:26:33.875773 I | etcdmain: Git SHA: 27fc7e2
2019-08-29 09:26:33.875779 I | etcdmain: Go Version: go1.10.4
2019-08-29 09:26:33.875788 I | etcdmain: Go OS/Arch: linux/amd64
2019-08-29 09:26:33.875796 I | etcdmain: setting maximum number of CPUs to 4, total number of available CPUs is 4
2019-08-29 09:26:33.876992 I | embed: peerTLS: cert = /etc/kubernetes/ssl/kube-etcd-192-168-9-63.pem, key = /etc/kube63-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true, crl-file =
2019-08-29 09:26:33.877559 I | embed: listening for peers on https://0.0.0.0:2380
2019-08-29 09:26:33.877593 I | embed: listening for client requests on 0.0.0.0:2379
2019-08-29 09:26:33.880816 I | etcdserver: name = etcd-k8s2-default-master
2019-08-29 09:26:33.880837 I | etcdserver: data dir = /var/lib/rancher/etcd/
2019-08-29 09:26:33.880845 I | etcdserver: member dir = /var/lib/rancher/etcd/member
2019-08-29 09:26:33.880854 I | etcdserver: heartbeat = 500ms
2019-08-29 09:26:33.880862 I | etcdserver: election = 5000ms
2019-08-29 09:26:33.880869 I | etcdserver: snapshot count = 100000
2019-08-29 09:26:33.880881 I | etcdserver: advertise client URLs = https://192.168.9.63:2379,https://192.168.9.63:400
2019-08-29 09:26:33.880891 I | etcdserver: initial advertise peer URLs = https://192.168.9.63:2380
2019-08-29 09:26:33.880902 I | etcdserver: initial cluster = etcd-k8s2-default-master=https://192.168.9.63:2380
2019-08-29 09:26:33.883288 I | etcdserver: starting member 2cb397d0a941ec03 in cluster ceb54976199a1ad4
2019-08-29 09:26:33.883320 I | raft: 2cb397d0a941ec03 became follower at term 0
2019-08-29 09:26:33.883335 I | raft: newRaft 2cb397d0a941ec03 [peers: [], term: 0, commit: 0, applied: 0, lastindex:
2019-08-29 09:26:33.883347 I | raft: 2cb397d0a941ec03 became follower at term 1
2019-08-29 09:26:33.886344 W | auth: simple token is not cryptographically signed
2019-08-29 09:26:33.888278 I | etcdserver: starting server... [version: 3.3.10, cluster version: to_be_decided]
2019-08-29 09:26:33.888576 I | etcdserver: 2cb397d0a941ec03 as single-node; fast-forwarding 9 ticks (election ticks 1
2019-08-29 09:26:33.888789 I | etcdserver/membership: added member 2cb397d0a941ec03 [https://192.168.9.63:2380] to cl
2019-08-29 09:26:33.890173 I | embed: ClientTLS: cert = /etc/kubernetes/ssl/kube-etcd-192-168-9-63.pem, key = /etc/ku9-63-key.pem, ca = , trusted-ca = /etc/kubernetes/ssl/kube-ca.pem, client-cert-auth = true, crl-file =
2019-08-29 09:26:35.883730 I | raft: 2cb397d0a941ec03 is starting a new election at term 1
2019-08-29 09:26:35.883765 I | raft: 2cb397d0a941ec03 became candidate at term 2
2019-08-29 09:26:35.883796 I | raft: 2cb397d0a941ec03 received MsgVoteResp from 2cb397d0a941ec03 at term 2
2019-08-29 09:26:35.883811 I | raft: 2cb397d0a941ec03 became leader at term 2
2019-08-29 09:26:35.883819 I | raft: raft.node: 2cb397d0a941ec03 elected leader 2cb397d0a941ec03 at term 2
2019-08-29 09:26:35.884055 I | etcdserver: setting up the initial cluster version to 3.3
2019-08-29 09:26:35.884163 I | embed: ready to serve client requests
2019-08-29 09:26:35.884190 I | etcdserver: published {Name:etcd-k8s2-default-master ClientURLs:[https://192.168.9.63:} to cluster ceb54976199a1ad4
2019-08-29 09:26:35.884361 N | etcdserver/membership: set the initial cluster version to 3.3
2019-08-29 09:26:35.884410 I | etcdserver/api: enabled capabilities for version 3.3
2019-08-29 09:26:35.885922 I | embed: serving client requests on [::]:2379
proto: no coders for int
proto: no encoder for ValueSize int [GetProperties]
2019-08-29 09:27:05.358509 W | etcdserver: read-only range request "key:"/registry/serviceaccounts/kube-public/defaunse_count:1 size:181" took too long (131.911897ms) to execute
2019-08-29 09:27:05.498503 W | etcdserver: read-only range request "key:"/registry/serviceaccounts/kube-node-lease/desponse_count:1 size:189" took too long (204.659952ms) to execute
2019-08-29 09:27:05.501293 W | etcdserver: read-only range request "key:"/registry/serviceaccounts/default/default"count:1 size:173" took too long (140.657181ms) to execute
2019-08-29 09:36:42.019985 I | mvcc: store.index: compact 1170
2019-08-29 09:36:42.021747 I | mvcc: finished scheduled compaction at 1170 (took 1.156378ms)
2019-08-29 09:37:08.941870 I | embed: rejected connection from "192.168.9.63:46038" (error "EOF", ServerName "")
2019-08-29 09:37:38.483173 I | embed: rejected connection from "192.168.9.63:56492" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
2019-08-29 09:37:43.611036 I | embed: rejected connection from "192.168.9.63:56514" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
2019-08-29 09:37:48.743188 I | embed: rejected connection from "192.168.9.63:56534" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
2019-08-29 09:40:45.748938 I | embed: rejected connection from "192.168.9.63:42007" (error "EOF", ServerName "")
2019-08-29 09:40:59.687044 I | embed: rejected connection from "192.168.9.63:57124" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
2019-08-29 09:41:04.819969 I | embed: rejected connection from "192.168.9.63:57140" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
2019-08-29 09:41:09.951009 I | embed: rejected connection from "192.168.9.63:57160" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
2019-08-29 09:41:42.028565 I | mvcc: store.index: compact 1610
2019-08-29 09:41:42.029735 I | mvcc: finished scheduled compaction at 1610 (took 712.864碌s)
2019-08-29 09:46:42.050216 I | mvcc: store.index: compact 2050
2019-08-29 09:46:42.052342 I | mvcc: finished scheduled compaction at 2050 (took 1.743065ms)
2019-08-29 09:51:42.053520 I | mvcc: store.index: compact 2491
2019-08-29 09:51:42.054932 I | mvcc: finished scheduled compaction at 2491 (took 898.152碌s)
2019-08-29 09:56:42.056564 I | mvcc: store.index: compact 2931
2019-08-29 09:56:42.057588 I | mvcc: finished scheduled compaction at 2931 (took 635.85碌s)
2019-08-29 10:01:42.059406 I | mvcc: store.index: compact 3371
2019-08-29 10:01:42.060648 I | mvcc: finished scheduled compaction at 3371 (took 864.209碌s)
2019-08-29 10:03:23.479778 I | embed: rejected connection from "192.168.9.63:39008" (error "EOF", ServerName "")
2019-08-29 10:03:37.356317 I | embed: rejected connection from "192.168.9.63:32994" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
2019-08-29 10:03:42.503829 I | embed: rejected connection from "192.168.9.63:33018" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
2019-08-29 10:03:47.647374 I | embed: rejected connection from "192.168.9.63:33032" (error "tls: failed to verify cliicate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candube-ca")", ServerName "")
MartinYangTW
on 29 Aug 2019
@superseb why that issue is closed? I am having the same problem with RKE v1.1.11.
Do you have any solution that i have missed in that issue?
bilalcaliskan
on 14 Nov 2020
Here is the workaround for that problem "etcdmain: rejected connection from "xxx:xxx" (error "remote error: tls: bad certificate", ServerName "")"
Run sudo rm -rf /etc/kubernetes/ /var/lib/kubelet/ /var/lib/etcd/ on your control plane nodes.
bilalcaliskan
on 14 Nov 2020
👍5
Was this page helpful?
0 / 5 - 0 ratings
Related issues
iljaweis
路
20Comments
gknepper
路
19Comments
superseb
路
16Comments
PDQDakota
路
15Comments
piwi91
路
25Comments
Most helpful comment
Here is the workaround for that problem "etcdmain: rejected connection from "xxx:xxx" (error "remote error: tls: bad certificate", ServerName "")"
Run sudo rm -rf /etc/kubernetes/ /var/lib/kubelet/ /var/lib/etcd/ on your control plane nodes.