Minikube: dnsmasq: during machine creation: Machine didn't return an IP after 120 seconds

Created on 9 Dec 2018  ยท  12Comments  ยท  Source: kubernetes/minikube

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG

Please provide the following details:

Environment:

Minikube version : v0.30.0

  • OS (e.g. from /etc/os-release):
[root@archserver ~]# cat /etc/os-release
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
ID_LIKE=archlinux
ANSI_COLOR="0;36"
HOME_URL="https://www.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"

  • VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName):
    "DriverName": "kvm2",
  • ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION):
 "Boot2DockerURL": "file:///home/cryto/.minikube/cache/iso/minikube-v0.30.0.iso",
        "ISO": "/home/cryto/.minikube/machines/minikube/boot2docker.iso",
  • Install tools: yay -S kubectl-bin minikube-bin docker-machine-driver-kvm2
  • Others:
[root@archserver ~]# virsh version
Compiled against library: libvirt 4.9.0
Using library: libvirt 4.9.0
Using API: QEMU 4.9.0
Running hypervisor: QEMU 3.0.0

[root@archserver ~]# dnsmasq --version
Dnsmasq version 2.80  Copyright (c) 2000-2018 Simon Kelley
Compile time options: IPv6 GNU-getopt DBus i18n IDN2 DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSE-detect inotify dumpfile

This software comes with ABSOLUTELY NO WARRANTY.
Dnsmasq is free software, and you are welcome to redistribute it
under the terms of the GNU General Public License, version 2 or 3.

โ— libvirtd.service - Virtualization daemon
   Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: disabled)
   Active: active (running) since Tue 2018-12-04 22:04:52 UTC; 4 days ago
     Docs: man:libvirtd(8)
           https://libvirt.org
 Main PID: 440 (libvirtd)
    Tasks: 23 (limit: 32768)
   Memory: 52.3M
   CGroup: /system.slice/libvirtd.service
           โ”œโ”€ 440 /usr/bin/libvirtd
           โ”œโ”€ 734 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/docker-machines.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
           โ”œโ”€ 735 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/docker-machines.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
           โ”œโ”€ 764 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
           โ”œโ”€ 765 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
           โ”œโ”€9801 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/minikube-net.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper
           โ””โ”€9802 /usr/bin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/minikube-net.conf --leasefile-ro --dhcp-script=/usr/lib/libvirt/libvirt_leaseshelper



md5-a5c6d7131f743a788068abaf44eb75a5



[root@archserver ~]# virsh net-list
 Name              State    Autostart   Persistent
----------------------------------------------------
 default           active   yes         yes
 docker-machines   active   yes         yes
 minikube-net      active   yes         yes



md5-a5c6d7131f743a788068abaf44eb75a5



[root@archserver ~]# ls /var/lib/libvirt/dnsmasq
default.addnhosts  default.hostsfile          docker-machines.conf       minikube-net.addnhosts  minikube-net.hostsfile  virbr0.status  virbr1.status  virbr2.status
default.conf       docker-machines.addnhosts  docker-machines.hostsfile  minikube-net.conf       virbr0.macs             virbr1.macs    virbr2.macs
[root@archserver ~]# cat /var/lib/libvirt/dnsmasq/virbr2.status
[root@archserver ~]# cat /var/lib/libvirt/dnsmasq/virbr1.status
[root@archserver ~]# cat /var/lib/libvirt/dnsmasq/virbr0.status
[root@archserver ~]# brctl show
bridge name     bridge id               STP enabled     interfaces
docker0         8000.02422574b795       no
virbr0          8000.5254004c2a7a       yes             virbr0-nic
                                                        vnet0
virbr1          8000.525400f245f1       yes             virbr1-nic
virbr2          8000.5254001a2828       yes             virbr2-nic
                                                        vnet1



md5-20d46f50e3d53a980963fb37daddfad1



[cryto@archserver ~]$ minikube start --vm-driver kvm2 --v=10 --alsologtostderr
I1209 09:55:49.814065    9761 notify.go:121] Checking for updates...
I1209 09:55:50.049275    9761 start.go:99] Viper configuration:
Aliases:
map[string]string{}
Override:
map[string]interface {}{"v":"10", "alsologtostderr":"true"}
PFlags:
map[string]viper.FlagValue{"kvm-network":viper.pflagValue{flag:(*pflag.Flag)(0xc420025ea0)}                                                                                                                                                  , "nfs-shares-root":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a80a0)}, "registry-mirror":v                                                                                                                                                  iper.pflagValue{flag:(*pflag.Flag)(0xc4203a85a0)}, "profile":viper.pflagValue{flag:(*pflag.                                                                                                                                                  Flag)(0xc420024dc0)}, "host-only-cidr":viper.pflagValue{flag:(*pflag.Flag)(0xc420025d60)},                                                                                                                                                   "hyperkit-vpnkit-sock":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8aa0)}, "iso-url":viper.                                                                                                                                                  pflagValue{flag:(*pflag.Flag)(0xc420025a40)}, "network-plugin":viper.pflagValue{flag:(*pfla                                                                                                                                                  g.Flag)(0xc4203a8780)}, "apiserver-ips":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a83c0)},                                                                                                                                                   "apiserver-name":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8280)}, "insecure-registry":v                                                                                                                                                  iper.pflagValue{flag:(*pflag.Flag)(0xc4203a8500)}, "kubernetes-version":viper.pflagValue{fl                                                                                                                                                  ag:(*pflag.Flag)(0xc4203a86e0)}, "dns-domain":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8                                                                                                                                                  460)}, "gpu":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8be0)}, "hyperkit-vsock-ports":vip                                                                                                                                                  er.pflagValue{flag:(*pflag.Flag)(0xc4203a8b40)}, "nfs-share":viper.pflagValue{flag:(*pflag.                                                                                                                                                  Flag)(0xc4203a8000)}, "apiserver-names":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8320)},                                                                                                                                                   "container-runtime":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8640)}, "cpus":viper.pflag                                                                                                                                                  Value{flag:(*pflag.Flag)(0xc420025c20)}, "disk-size":viper.pflagValue{flag:(*pflag.Flag)(0x                                                                                                                                                  c420025cc0)}, "uuid":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8a00)}, "xhyve-disk-driver                                                                                                                                                  ":viper.pflagValue{flag:(*pflag.Flag)(0xc420025f40)}, "vm-driver":viper.pflagValue{flag:(*p                                                                                                                                                  flag.Flag)(0xc420025ae0)}, "cache-images":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a88c0)                                                                                                                                                  }, "docker-opt":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a81e0)}, "mount":viper.pflagValu                                                                                                                                                  e{flag:(*pflag.Flag)(0xc420025860)}, "mount-string":viper.pflagValue{flag:(*pflag.Flag)(0xc                                                                                                                                                  420025900)}, "docker-env":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8140)}, "feature-gate                                                                                                                                                  s":viper.pflagValue{flag:(*pflag.Flag)(0xc4203a8820)}, "keep-context":viper.pflagValue{flag                                                                                                                                                  :(*pflag.Flag)(0xc4200257c0)}, "disable-driver-mounts":viper.pflagValue{flag:(*pflag.Flag)(                                                                                                                                                  0xc4200259a0)}, "hyperv-virtual-switch":viper.pflagValue{flag:(*pflag.Flag)(0xc420025e00)},                                                                                                                                                   "bootstrapper":viper.pflagValue{flag:(*pflag.Flag)(0xc420024e60)}, "extra-config":viper.pf                                                                                                                                                  lagValue{flag:(*pflag.Flag)(0xc4203a8960)}, "memory":viper.pflagValue{flag:(*pflag.Flag)(0x                                                                                                                                                  c420025b80)}}
Env:
map[string]string{}
Key/Value Store:
map[string]interface {}{}
Config:
map[string]interface {}{"wantreporterror":true}
Defaults:
map[string]interface {}{"wantupdatenotification":true, "reminderwaitperiodinhours":24, "als                                                                                                                                                  ologtostderr":"false", "log_dir":"", "wantreporterror":false, "wantreporterrorprompt":true,                                                                                                                                                   "wantkubectldownloadmsg":true, "wantnonedriverwarning":true, "showdriverdeprecationnotific                                                                                                                                                  ation":true, "showbootstrapperdeprecationnotification":true, "v":"0"}
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
I1209 09:55:50.050000    9761 utils.go:100] retry loop 0
I1209 09:55:50.050043    9761 cluster.go:69] Machine does not exist... provisioning new mac                                                                                                                                                  hine
I1209 09:55:50.050073    9761 cluster.go:70] Provisioning machine with config: {MinikubeISO                                                                                                                                                  :https://storage.googleapis.com/minikube/iso/minikube-v0.30.0.iso Memory:2048 CPUs:2 DiskSi                                                                                                                                                  ze:20000 VMDriver:kvm2 HyperkitVpnKitSock: HyperkitVSockPorts:[] XhyveDiskDriver:ahci-hd Do                                                                                                                                                  ckerEnv:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtual                                                                                                                                                  Switch: KvmNetwork:default Downloader:{} DockerOpt:[] DisableDriverMounts:false NFSShare:[]                                                                                                                                                   NFSSharesRoot:/nfsshares UUID: GPU:false}
I1209 09:55:50.050202    9761 downloader.go:56] Not caching ISO, using https://storage.goog                                                                                                                                                  leapis.com/minikube/iso/minikube-v0.30.0.iso
Found binary path at /usr/bin/docker-machine-driver-kvm2
Launching plugin server for driver kvm2
Plugin server listening at address 127.0.0.1:39359
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetMachineName
(minikube) Calling .DriverName
Reading certificate data from /home/cryto/.minikube/certs/ca.pem
Decoding PEM data...
Parsing certificate...
Reading certificate data from /home/cryto/.minikube/certs/cert.pem
Decoding PEM data...
Parsing certificate...
Running pre-create checks...
(minikube) Calling .PreCreateCheck
(minikube) Calling .GetConfigRaw
Creating machine...
(minikube) Calling .Create
(minikube) Creating machine...
(minikube) Creating network...
(minikube) Setting up minikube home directory...
(minikube) DBG | Setting executable bit set on /home/cryto/.minikube/machines/minikube
(minikube) DBG | Setting executable bit set on /home/cryto/.minikube/machines
(minikube) DBG | Setting executable bit set on /home/cryto/.minikube
(minikube) Building disk image...
(minikube) DBG | Setting executable bit set on /home
(minikube) Downloading /home/cryto/.minikube/cache/boot2docker.iso from file:///home/cryto/                                                                                                                                                  .minikube/cache/iso/minikube-v0.30.0.iso...
(minikube) DBG | Writing magic tar header
(minikube) DBG | Writing SSH key tar header
(minikube) Creating domain...
(minikube) DBG | Finished creating machine, now starting machine...
(minikube) Creating network...
(minikube) Ensuring networks are active...
(minikube) Ensuring network default is active
(minikube) Ensuring network minikube-net is active
(minikube) Getting domain xml...
(minikube) Creating domain...
(minikube) Waiting to get IP...
(minikube) DBG | Waiting for machine to come up 0/40
(minikube) DBG | Waiting for machine to come up 1/40
(minikube) DBG | Waiting for machine to come up 2/40
(minikube) DBG | Waiting for machine to come up 3/40
(minikube) DBG | Waiting for machine to come up 4/40
(minikube) DBG | Waiting for machine to come up 5/40
(minikube) DBG | Waiting for machine to come up 6/40
(minikube) DBG | Waiting for machine to come up 7/40
(minikube) DBG | Waiting for machine to come up 8/40
(minikube) DBG | Waiting for machine to come up 9/40
(minikube) DBG | Waiting for machine to come up 10/40
(minikube) DBG | Waiting for machine to come up 11/40
(minikube) DBG | Waiting for machine to come up 12/40
(minikube) DBG | Waiting for machine to come up 13/40
(minikube) DBG | Waiting for machine to come up 14/40
(minikube) DBG | Waiting for machine to come up 15/40
(minikube) DBG | Waiting for machine to come up 16/40
(minikube) DBG | Waiting for machine to come up 17/40
(minikube) DBG | Waiting for machine to come up 18/40
(minikube) DBG | Waiting for machine to come up 19/40
(minikube) DBG | Waiting for machine to come up 20/40
(minikube) DBG | Waiting for machine to come up 21/40
(minikube) DBG | Waiting for machine to come up 22/40
(minikube) DBG | Waiting for machine to come up 23/40
(minikube) DBG | Waiting for machine to come up 24/40
(minikube) DBG | Waiting for machine to come up 25/40
(minikube) DBG | Waiting for machine to come up 26/40
(minikube) DBG | Waiting for machine to come up 27/40
(minikube) DBG | Waiting for machine to come up 28/40
(minikube) DBG | Waiting for machine to come up 29/40
(minikube) DBG | Waiting for machine to come up 30/40
(minikube) DBG | Waiting for machine to come up 31/40
(minikube) DBG | Waiting for machine to come up 32/40
(minikube) DBG | Waiting for machine to come up 33/40
(minikube) DBG | Waiting for machine to come up 34/40
(minikube) DBG | Waiting for machine to come up 35/40
(minikube) DBG | Waiting for machine to come up 36/40
(minikube) DBG | Waiting for machine to come up 37/40
(minikube) DBG | Waiting for machine to come up 38/40
(minikube) DBG | Waiting for machine to come up 39/40
(minikube) DBG | Waiting for machine to come up 40/40
E1209 09:57:58.546098    9761 start.go:168] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds.

Anything else do we need to know:
The dnsmasq status file for minikube-net at /var/lib/libvirt/dnsmasq/virbr2.status is empty! Actually all three status files for the three available virtual networks (default - virbr0.status, docker-machines - virbr1.status, minikube-net - virbr2.status) are empty. I think i got the same problem with minishift with the kvm driver (not kvm2) and it has been reported minishift#3028 - but still no solution though

I think the created VM has troubles to lease IP from dnsmasq but i dont know how to get it solved. Thanks you!

causvm-networking ckvm2 help wanted prioritawaiting-more-evidence

Most helpful comment

The docker-machine-driver-kvm2 is on the minikube release page for v0.31.0:
https://github.com/kubernetes/minikube/releases/tag/v0.31.0

Here is a direct link:
https://github.com/kubernetes/minikube/releases/download/v0.31.0/docker-machine-driver-kvm2

All 12 comments

It looks like this strange bug was added by me when I tried to fix a problem with dnsmasq in #2745.
I don't know why, but on some machines the VM cannot get the IP from dnsmasq dhcp when this setting is enabled. I tried to discover the problem by tcpdumping the bridge, but I don't even see dhcp requests from VM. Anyway, I will try to fix it.
You could try to use the driver binary from 0.25.0 version, it should work for you.

I was able to get minikube v0.33.1 and docker-machine-driver-kvm2 v0.31.0 to work correctly on CentOS 7. More recent versions of docker-machine-driver-kvm2 don't work for me.

Link to docker-machine-driver-kvm2 v0.31.0 to share ? No track found. Thanks

The docker-machine-driver-kvm2 is on the minikube release page for v0.31.0:
https://github.com/kubernetes/minikube/releases/tag/v0.31.0

Here is a direct link:
https://github.com/kubernetes/minikube/releases/download/v0.31.0/docker-machine-driver-kvm2

I have the same problem with kvm2 and kvm drivers. Also i check new version 0.34.1 and version 0.25.0 it's didn't work for me ;(

If i can give you some information tell me what to do.

@aelsergeev did you solve this problem? I am facing the same.

@aelsergeev did you solve this problem? I am facing the same.

No, I use VB driver now, you can better to write about the problem in this issue https://github.com/kubernetes/minikube/issues/3566

@aelsergeev it is fixed with the latest version of minikube. Will fail once you minikube stop && minikube delete && rm -rf ~/.minikube && rm -rf /tmp/*

latest version not help me

@aelsergeev i am using debian.

I'm closing this issue as it hasn't seen activity in awhile, and it's unclear if this issue still exists. If this issue does continue to exist in the most recent release of minikube, please feel free to re-open it.

Do make sure to upgrade first to the latest kvm driver:

curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2 && sudo install docker-machine-driver-kvm2 /usr/local/bin/

Thank you for opening the issue!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

xmnlab picture xmnlab  ยท  3Comments

olalonde picture olalonde  ยท  3Comments

jroucheton picture jroucheton  ยท  3Comments

kphatak picture kphatak  ยท  3Comments

alber70g picture alber70g  ยท  3Comments