Rke: RKE SSH: failed to connect to the following etcd host

[armanriazi]

I saw https://github.com/yamamoto-febc/terraform-provider-rke/issues/9 and I checked ssh to ip 10,15,16 but diden't problem with openssl-server when I run this yaml file I get some error on etcd,ssh
RKE 0.1.15

Docker version: 18.06

Operating system and kernel: Ubuntu 18.04 LTS

Type/provider of hosts: VMWare

cluster.yml

If you intened to deploy Kubernetes in an air-gapped environment,

please consult the documentation on how to configure custom RKE images.

nodes:
- address: 172.18.3.10
  port: "22"
  internal_address: 172.18.3.10
  role:
  - controlplane
  - etcd
  hostname_override: 172.18.3.10
  user: rke
  docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: /home/rke/.ssh/id_rsa
  labels: {}
- address: 172.18.3.15
  port: "22"
  internal_address: 172.18.3.15
  role:
  - controlplane
  - worker
  - etcd
  hostname_override: 172.18.3.15
  user: rke
  docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: ~/.ssh/id_rsa
  labels: {}
- address: 172.18.3.16
  port: "22"
  internal_address: 172.18.3.16
  role:
  - controlplane
  - worker
  - etcd
  hostname_override: 172.18.3.16
  user: rke
  docker_socket: /var/run/docker.sock
  ssh_key: ""
  ssh_key_path: /home/rke/.ssh/id_rsa
  labels: {}
services:
  etcd:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    external_urls: []
    ca_cert: ""
    cert: ""
    key: ""
    path: ""
    snapshot: null
    retention: ""
    creation: ""
  kube-api:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    service_cluster_ip_range: 192.168.0.0/16
    service_node_port_range: ""
    pod_security_policy: false
  kube-controller:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    cluster_cidr: 192.169.0.0/16
    service_cluster_ip_range:  192.168.0.0/16
  scheduler:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
  kubelet:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    cluster_domain: cluster.local
    infra_container_image: ""
    cluster_dns_server: 192.168.0.1
    fail_swap_on: false
  kubeproxy:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
network:
  plugin: calico
  options: {}
authentication:
  strategy: x509
  options: {}
  sans: []
addons: ""
addons_include:
- http://172.18.3.10/manifest.yaml
system_images:
  etcd: 172.18.3.9:5000/rancher/coreos-etcd:v3.2.18
  alpine: 172.18.3.9:5000/rancher/rke-tools:v0.1.15
  nginx_proxy: 172.18.3.9:5000/rancher/rke-tools:v0.1.15
  cert_downloader: 172.18.3.9:5000/rancher/rke-tools:v0.1.15
  kubernetes_services_sidecar: 172.18.3.9:5000/rancher/rke-tools:v0.1.15
  kubedns: 172.18.3.9:5000/rancher/k8s-dns-kube-dns-amd64:1.14.10
  dnsmasq: 172.18.3.9:5000/rancher/k8s-dns-dnsmasq-nanny-amd64:1.14.10
  kubedns_sidecar: 172.18.3.9:5000/rancher/k8s-dns-sidecar-amd64:1.14.10
  kubedns_autoscaler: 172.18.3.9:5000/rancher/cluster-proportional-autoscaler-amd64:1.0.0
  kubernetes: 172.18.3.9:5000/rancher/hyperkube:v1.11.6-rancher1
  flannel: 172.18.3.9:5000/rancher/coreos-flannel:v0.10.0
  flannel_cni: 172.18.3.9:5000/rancher/coreos-flannel-cni:v0.3.0
  calico_node: 172.18.3.9:5000/rancher/calico-node:v3.1.3
  calico_cni: 172.18.3.9:5000/rancher/calico-cni:v3.1.3
  calico_controllers: ""
  calico_ctl: 172.18.3.9:5000/rancher/calico-ctl:v2.0.0
  canal_node: 172.18.3.9:5000/rancher/calico-node:v3.1.3
  canal_cni: 172.18.3.9:5000/rancher/calico-cni:v3.1.3
  canal_flannel: 172.18.3.9:5000/rancher/coreos-flannel:v0.10.0
  wave_node: 172.18.3.9:5000/weaveworks/weave-kube:2.1.2
  weave_cni: 172.18.3.9:5000/weaveworks/weave-npc:2.1.2
  pod_infra_container: 172.18.3.9:5000/rancher/pause-amd64:3.1
  ingress: 172.18.3.9:5000/rancher/nginx-ingress-controller:0.16.2-rancher1
  ingress_backend: 172.18.3.9:5000/rancher/nginx-ingress-controller-defaultbackend:1.4
  metrics_server: 172.18.3.9:5000/rancher/metrics-server-amd64:v0.2.1
ssh_key_path: /home/rke/.ssh/id_rsa
ssh_agent_auth: true
authorization:
  mode: rbac
  options: {}
ignore_docker_version: false
kubernetes_version: ""
private_registries:
- url: <172.18.3.9:5000> # private registry url
  is_default: true
ingress:
  provider: ""
  options: {}
  node_selector: {}
  extra_args: {}
cluster_name: ""
cloud_provider:
  name: ""
prefix_path: ""
addon_job_timeout: 0
bastion_host:
  address: ""
  port: ""
  user: ""
  ssh_key: ""
  ssh_key_path: ""
monitoring:
  provider: ""
  options: {}

Steps to Reproduce:
sudo rke up --config ./myrancher/cluster.yml

Results:

INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [172.18.3.15]
WARN[0000] Failed to set up SSH tunneling for host [172.18.3.15]: Can't retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [172.18.3.15:22] using SSH. Please check if the configured key or specified key file is a valid SSH Private Key. Error: Error configuring SSH: ssh: no key found
INFO[0000] [dialer] Setup tunnel for host [172.18.3.16]
WARN[0000] Failed to set up SSH tunneling for host [172.18.3.16]: Can't retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [172.18.3.16:22] using SSH. Please check if the configured key or specified key file is a valid SSH Private Key. Error: Error configuring SSH: ssh: no key found
INFO[0000] [dialer] Setup tunnel for host [172.18.3.10]
WARN[0000] Failed to set up SSH tunneling for host [172.18.3.10]: Can't retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [172.18.3.10:22] using SSH. Please check if the configured key or specified key file is a valid SSH Private Key. Error: Error configuring SSH: ssh: no key found
WARN[0000] Removing host [172.18.3.15] from node lists
WARN[0000] Removing host [172.18.3.16] from node lists
WARN[0000] Removing host [172.18.3.10] from node lists
FATA[0000] Cluster must have at least one etcd plane host: failed to connect to the following etcd host(s) [172.18.3.15]

[superseb]

Can you check https://rancher.com/docs/rke/v0.1.x/en/troubleshooting/ssh-connectivity-errors/#failed-to-dial-ssh-using-address-xxx-xxx-xxx-xxx-xx-error-configuring-ssh-ssh-no-key-found

[superseb]

Let me know if you are still experiencing issues.

[renve]

went and checked the ssh per your link above and it checked out fine. My rke cluster file
nodes:
I tried changing the address to the public ip as well and it does not work. I tried using only the private ip as the address only. I checked that ubuntu has access to docker commands.

• address: 10.0.1.162
  internal_address: 10.0.1.162
  user: ubuntu
  role:
  
  
  
  • controlplane
  
  
  • etcd
    
    ssh_key_path: ~/.ssh/id_rsa
  
  
• address: 10.0.1.113
  internal_address: 10.0.1.113
  user: ubuntu
  role:
  
  
  
  • controlplane
  
  
  • etcd
    
    ssh_key_path: ~/.ssh/id_rsa
    
    cluster_name: myname
  
  

ssh_agent_auth: true

services:
etcd:
snapshot: false

I get these errors

INFO[0000] Running RKE version: v0.3.2
INFO[0000] Initiating Kubernetes cluster
INFO[0000] [certificates] Generating Kubernetes API server certificates
INFO[0000] [certificates] Generating admin certificates and kubeconfig
INFO[0000] [certificates] Generating etcd-10.0.1.162 certificate and key
INFO[0000] [certificates] Generating etcd-10.0.1.113 certificate and key
INFO[0000] Successfully Deployed state file at [./rancher-cluster.rkestate]
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [10.0.1.162]
INFO[0000] [dialer] Setup tunnel for host [10.0.1.113]
WARN[0000] Failed to set up SSH tunneling for host [10.0.1.162]: Can't retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [10.0.1.162:22] using SSH. Please check if the configured key or specified key file is a valid SSH Private Key. Error: Error configuring SSH: ssh: no key found
WARN[0000] Failed to set up SSH tunneling for host [10.0.1.113]: Can't retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [10.0.1.113:22] using SSH. Please check if the configured key or specified key file is a valid SSH Private Key. Error: Error configuring SSH: ssh: no key found
WARN[0000] Removing host [10.0.1.162] from node lists
WARN[0000] Removing host [10.0.1.113] from node lists
FATA[0000] Cluster must have at least one etcd plane host: failed to connect to the following etcd host(s) [10.0.1.162]
ubuntu@master1:~$ chmod 600 .ssh/id_rsa
ubuntu@master1:~$ rke up --config rancher-cluster.yaml
INFO[0000] Running RKE version: v0.3.2
INFO[0000] Initiating Kubernetes cluster
INFO[0000] [certificates] Generating admin certificates and kubeconfig
INFO[0000] Successfully Deployed state file at [./rancher-cluster.rkestate]
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [10.0.1.113]
WARN[0000] Failed to set up SSH tunneling for host [10.0.1.113]: Can't retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [10.0.1.113:22] using SSH. Please check if the configured key or specified key file is a valid SSH Private Key. Error: Error configuring SSH: ssh: no key found
INFO[0000] [dialer] Setup tunnel for host [10.0.1.162]
WARN[0000] Failed to set up SSH tunneling for host [10.0.1.162]: Can't retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [10.0.1.162:22] using SSH. Please check if the configured key or specified key file is a valid SSH Private Key. Error: Error configuring SSH: ssh: no key found
WARN[0000] Removing host [10.0.1.113] from node lists
WARN[0000] Removing host [10.0.1.162] from node lists
FATA[0000] Cluster must have at least one etcd plane host: failed to connect to the following etcd host(s) [10.0.1.113]