Lxd: LXD in VMware appears to have issues connecting to physnic

[kp3nguinz]

The template below is mostly useful for bug reports and support questions.
Feel free to remove anything which doesn't apply to you and add more information where it makes sense.

Required information

• Distribution: Ubuntu
• Distribution version: 16.04.1

• The output of "lxc info" or if that fails:
  `apicompat: 0
  auth: trusted
  environment:
  addresses: []
  architectures:

  • x86_64
  • i686
    certificate: |
    

  driver: lxc
  driverversion: 2.0.3
  kernel: Linux
  kernelarchitecture: x86_64
  kernelversion: 4.4.0-31-generic
  server: lxd
  serverpid: 4293
  serverversion: 2.0.3
  storage: zfs
  storageversion: "5"
  config:
  storage.zfs_pool_name: logging
  public: false
  `

  Issue description

  macvlan doesnt appear to allow access to physical interface. In general I have tried about 10 guides, and cannot seem to get containers direct access to the network in order to pull a DHCP address from my network's DHCP. I have tried bridged mode as well.

A brief description of what failed or what could be improved.
Easy method for having containers pretend they are physical boxes accessing the network.

Steps to reproduce

Update your server: sudo apt-get update ; sudo apt-get dist-upgrade -y ; sudo apt-get install lxd -y
Update your groups: newgrp
Install ZFS (it's better that you do, not required): sudo apt-get install zfs -yf
Install bridge-utils: sudo apt-get install bridge-utils
Initialize LXD and configure it using wizard: sudo lxd init
Tell LXD to bridge containers to physical nic: lxc profile device set default eth0 parent ; lxc profile device set default eth0 nictype macvlan
Create containers service account sudo useradd -G lxd -s /bin/rbash
Change to service account: sudo su containers
Launch your container (Note: the hostname you choose will be it's hostname...): lxc launch ubuntu:16.04 mycontainerhostname

Information to attach

• [ ] any relevant kernel output (dmesg)
  [ 813.544723] IPv6: ADDRCONF(NETDEV_UP): ens192: link is not ready
  [ 837.741818] vmxnet3 0000:0b:00.0 ens192: intr type 3, mode 0, 3 vectors allocated
  [ 837.742708] vmxnet3 0000:0b:00.0 ens192: NIC Link is Up 10000 Mbps
  [ 860.439090] audit: type=1400 audit(1469742660.705:30): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lxd-unda-elsa_" pid=6899 comm="apparmor_parser"
  [ 860.571567] eth0: renamed from mc55FB5P
  [ 860.587109] device ens192 entered promiscuous mode
  [ 890.917963] device eth0 entered promiscuous mode
  [ 924.731516] device eth0 left promiscuous mode
• [ ] container log (lxc info NAME --show-log | less )
  Name: unda-elsa
  Architecture: x86_64
  Created: 2016/07/28 21:50 UTC
  Status: Running
  Type: persistent
  Profiles: default
  Pid: 6903
  Ips:
  eth0: inet6 fe80::216:3eff:fe56:d16b
  lo: inet 127.0.0.1
  lo: inet6 ::1
  Resources:
  Processes: 25
  Disk usage:
  root: 6.93MB
  Memory usage:
  Memory (current): 56.73MB
  Memory (peak): 179.48MB
  Network usage:
  lo:
  Bytes received: 264 bytes
  Bytes sent: 264 bytes
  Packets received: 4
  Packets sent: 4
  eth0:
  Bytes received: 1.04kB
  Bytes sent: 8.65kB
  Packets received: 7
  Packets sent: 32

Log:

        lxc 20160728145100.592 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 3
        lxc 20160728145100.592 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 9
        lxc 20160728145100.595 INFO     lxc_container - lxccontainer.c:do_lxcapi_start:797 - Attempting to set proc title to [lxc monitor] /var/lib/lxd/containers unda-elsa
        lxc 20160728145100.595 INFO     lxc_start - start.c:lxc_check_inherited:251 - closed inherited fd 9
        lxc 20160728145100.595 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
        lxc 20160728145100.596 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .reject_force_umount  # comment this to allow umount -f;  not recommended.
        lxc 20160728145100.596 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for reject_force_umount action 0
        lxc 20160728145100.596 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts

• [ ] main daemon log (/var/log/lxd.log)
• [ ] output of the client with --debug
• [ ] output of the daemon with --debug

[stgraber]

macvlan and straight bridging with VMWare usually won't work because of VMWare doing MAC filtering on their side. So any traffic coming from a MAC address other than the VM's.

[stgraber]

I believe there is a flag somewhere in recent vmware to turn off the MAC filtering feature at which point things should start to work normally.

[kp3nguinz]

@stgraber ugh... figured it was esxi specific... spent 27 hours on this, Thanks!

[hjabbour]

To be specific for those suffering its the vswitch configuration on ESXi make it promiscuous mode

[scottb76]

I also had this problem with a container running on a VM and it was immediately resolved by enabling promiscuous mode on the VMWare vSwitch. Thank you all of you, saved me hours no doubt!