Lxd: LXD new NIC logic fails to predict NIC name

Created on 10 Sep 2019  路  8Comments  路  Source: lxc/lxd

Note: no information is censored because this is a test instance in an isolated environment - it's all getting destroyed as soon as the issue is fixed.

Required information

  • Distribution: Alpine Linux
  • Distribution version: Edge
  • The output of "lxc info":
config: {}
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- macaroon_authentication
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- container_mount_propagation
- container_backup
- devlxd_images
- container_local_cross_pool_handling
- proxy_unix
- proxy_udp
- clustering_join
- proxy_tcp_udp_multi_port_handling
- network_state
- proxy_unix_dac_properties
- container_protection_delete
- unix_priv_drop
- pprof_http
- proxy_haproxy_protocol
- network_hwaddr
- proxy_nat
- network_nat_order
- container_full
- candid_authentication
- backup_compression
- candid_config
- nvidia_runtime_config
- storage_api_volume_snapshots
- storage_unmapped
- projects
- candid_config_key
- network_vxlan_ttl
- container_incremental_copy
- usb_optional_vendorid
- snapshot_scheduling
- container_copy_project
- clustering_server_address
- clustering_image_replication
- container_protection_shift
- snapshot_expiry
- container_backup_override_pool
- snapshot_expiry_creation
- network_leases_location
- resources_cpu_socket
- resources_gpu
- resources_numa
- kernel_features
- id_map_current
- event_location
- storage_api_remote_volume_snapshots
- network_nat_address
- container_nic_routes
- rbac
- cluster_internal_copy
- seccomp_notify
- lxc_features
- container_nic_ipvlan
- network_vlan_sriov
- storage_cephfs
- container_nic_ipfilter
- resources_v2
- container_exec_user_group_cwd
- container_syscall_intercept
- container_disk_shift
- storage_shifted
- resources_infiniband
- daemon_storage
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
environment:
  addresses: []
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----
    MIICCTCCAY+gAwIBAgIQLn+i9uv5Zs0Lx1twRZ0VNTAKBggqhkjOPQQDAzA2MRww
    GgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRYwFAYDVQQDDA1yb290QGx4ZC10
    ZXN0MB4XDTE5MDkwNjE5MzMwOVoXDTI5MDkwMzE5MzMwOVowNjEcMBoGA1UEChMT
    bGludXhjb250YWluZXJzLm9yZzEWMBQGA1UEAwwNcm9vdEBseGQtdGVzdDB2MBAG
    ByqGSM49AgEGBSuBBAAiA2IABDFJV9Oeg/YeuhVgtJbm0FtjpqtEWNOShstgk8U4
    FO3Oaf+SjSAFcZ0YP89RQtr2bqNfUfH01vUzhi3yzfcg3WIgq0g0DMZbkXUEw8Bh
    JOusaJKq/FXLPJTPi1jbZux1UaNiMGAwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM
    MAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwKwYDVR0RBCQwIoIIbHhkLXRlc3SH
    BJuKhsSHECABGfCwAQL7VAAC//5AOI0wCgYIKoZIzj0EAwMDaAAwZQIwBPrcDMhd
    aXxreL402xXOPIx8/QqWyu0GnGZIXh8B435hIpVgLKI6jzu8xHKUUm47AjEAmLxB
    DgpGfBNg5xdjs0jPuqZ5lXRD3C9QJh6YzcBJpCJwOUfk7dVU4J4QdWTpz/r+
    -----END CERTIFICATE-----
  certificate_fingerprint: d03a7cde11acb507305fd109b75b9c2628addd334a91c47fb849c7993e54e377
  driver: lxc
  driver_version: 3.2.1
  kernel: Linux
  kernel_architecture: x86_64
  kernel_features:
    netnsid_getifaddrs: "false"
    seccomp_listener: "false"
    shiftfs: "false"
    uevent_injection: "true"
    unpriv_fscaps: "true"
  kernel_version: 4.19.71-0-vanilla
  lxc_features:
    mount_injection_file: "true"
    network_gateway_device_route: "true"
    network_ipvlan: "true"
    network_l2proxy: "true"
    network_phys_macvlan_mtu: "true"
    seccomp_notify: "true"
  project: default
  server: lxd
  server_clustered: false
  server_name: lxd-test
  server_pid: 2333
  server_version: "3.17"
  storage: dir
  storage_version: "1"
  • Storage backend in use: dir (btrfs on another system)

Issue description

When starting a container with lxd >=3.16, the new NIC logic fails to identify the correct device to modify.

Steps to reproduce

  1. lxd init -> all defaults are fine
  2. lxc init images:alpine/edge test
  3. lxc start test => Error: Common start logic: Failed to start device 'eth0': Failed to set the MAC address: Failed to run: ip link set dev veth813be3d0 address 00:16:3e:67:1d:11: ip: ioctl 0x8933 failed: No such device
  4. ip link show => veth0@vethc344e738 and vethc344e738@veth0 - note the mismatch.

Information to attach

  • [x] Any relevant kernel output (dmesg)
    [95396.724688] IPv6: ADDRCONF(NETDEV_UP): vethc344e738: link is not ready
  • [x] Container log (lxc info NAME --show-log)
    ```
    Name: test
    Location: none
    Remote: unix://
    Architecture: x86_64
    Created: 2019/09/06 19:35 UTC
    Status: Stopped
    Type: persistent
    Profiles: default
    Log: 
 - [x] Container configuration (`lxc config show NAME --expanded`)
  ```
architecture: x86_64
config:
image.architecture: amd64
image.description: Alpine edge amd64 (20190906_13:00)
image.os: Alpine
image.release: edge
image.serial: "20190906_13:00"
volatile.apply_template: create
volatile.base_image: 92d2c7c63c95dcfdb6fe0c34af6573bad4d76f23e328343ff52f45b5dca64721
volatile.eth0.hwaddr: 00:16:3e:67:1d:11
volatile.idmap.base: "0"
volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk
ephemeral: false
profiles:
  - default                                                                                                                                     stateful: false
description: ""
  • [ ] Main daemon log (at /var/log/lxd/lxd.log or /var/snap/lxd/common/lxd/logs/lxd.log) empty
  • [x] Output of the client with --debug
    `` t=2019-09-10T13:36:53-0400 lvl=dbug msg="Connecting to a local LXD over a Unix socket" t=2019-09-10T13:36:53-0400 lvl=dbug msg="Sending request to LXD" etag= method=GET url=http://unix.socket/1.0 t=2019-09-10T13:36:53-0400 lvl=dbug msg="Got response struct from LXD" t=2019-09-10T13:36:53-0400 lvl=dbug msg="\n\t{\n\t\t\"config\": {},\n\t\t\"api_extensions\": [\n\t\t\t\"storage_zfs_remove_snapshots\",\n\t\t\t\"container_host_shutdown_timeout\",\n\t\t\t\"container_stop_priority\",\n\t\t\t\"container_syscall_filtering\",\n\t\t\t\"auth_pki\",\n\t\t\t\"container_last_used_at\",\n\t\t\t\"etag\",\n\t\t\t\"patch\",\n\t\t\t\"usb_devices\",\n\t\t\t\"https_allowed_credentials\",\n\t\t\t\"image_compression_algorithm\",\n\t\t\t\"directory_manipulation\",\n\t\t\t\"container_cpu_time\",\n\t\t\t\"storage_zfs_use_refquota\",\n\t\t\t\"storage_lvm_mount_options\",\n\t\t\t\"network\",\n\t\t\t\"profile_usedby\",\n\t\t\t\"container_push\",\n\t\t\t\"container_exec_recording\",\n\t\t\t\"certificate_update\",\n\t\t\t\"container_exec_signal_handling\",\n\t\t\t\"gpu_devices\",\n\t\t\t\"container_image_properties\",\n\t\t\t\"migration_progress\",\n\t\t\t\"id_map\",\n\t\t\t\"network_firewall_filtering\",\n\t\t\t\"network_routes\",\n\t\t\t\"storage\",\n\t\t\t\"file_delete\",\n\t\t\t\"file_append\",\n\t\t\t\"network_dhcp_expiry\",\n\t\t\t\"storage_lvm_vg_rename\",\n\t\t\t\"storage_lvm_thinpool_rename\",\n\t\t\t\"network_vlan\",\n\t\t\t\"image_create_aliases\",\n\t\t\t\"container_stateless_copy\",\n\t\t\t\"container_only_migration\",\n\t\t\t\"storage_zfs_clone_copy\",\n\t\t\t\"unix_device_rename\",\n\t\t\t\"storage_lvm_use_thinpool\",\n\t\t\t\"storage_rsync_bwlimit\",\n\t\t\t\"network_vxlan_interface\",\n\t\t\t\"storage_btrfs_mount_options\",\n\t\t\t\"entity_description\",\n\t\t\t\"image_force_refresh\",\n\t\t\t\"storage_lvm_lv_resizing\",\n\t\t\t\"id_map_base\",\n\t\t\t\"file_symlinks\",\n\t\t\t\"container_push_target\",\n\t\t\t\"network_vlan_physical\",\n\t\t\t\"storage_images_delete\",\n\t\t\t\"container_edit_metadata\",\n\t\t\t\"container_snapshot_stateful_migration\",\n\t\t\t\"storage_driver_ceph\",\n\t\t\t\"storage_ceph_user_name\",\n\t\t\t\"resource_limits\",\n\t\t\t\"storage_volatile_initial_source\",\n\t\t\t\"storage_ceph_force_osd_reuse\",\n\t\t\t\"storage_block_filesystem_btrfs\",\n\t\t\t\"resources\",\n\t\t\t\"kernel_limits\",\n\t\t\t\"storage_api_volume_rename\",\n\t\t\t\"macaroon_authentication\",\n\t\t\t\"network_sriov\",\n\t\t\t\"console\",\n\t\t\t\"restrict_devlxd\",\n\t\t\t\"migration_pre_copy\",\n\t\t\t\"infiniband\",\n\t\t\t\"maas_network\",\n\t\t\t\"devlxd_events\",\n\t\t\t\"proxy\",\n\t\t\t\"network_dhcp_gateway\",\n\t\t\t\"file_get_symlink\",\n\t\t\t\"network_leases\",\n\t\t\t\"unix_device_hotplug\",\n\t\t\t\"storage_api_local_volume_handling\",\n\t\t\t\"operation_description\",\n\t\t\t\"clustering\",\n\t\t\t\"event_lifecycle\",\n\t\t\t\"storage_api_remote_volume_handling\",\n\t\t\t\"nvidia_runtime\",\n\t\t\t\"container_mount_propagation\",\n\t\t\t\"container_backup\",\n\t\t\t\"devlxd_images\",\n\t\t\t\"container_local_cross_pool_handling\",\n\t\t\t\"proxy_unix\",\n\t\t\t\"proxy_udp\",\n\t\t\t\"clustering_join\",\n\t\t\t\"proxy_tcp_udp_multi_port_handling\",\n\t\t\t\"network_state\",\n\t\t\t\"proxy_unix_dac_properties\",\n\t\t\t\"container_protection_delete\",\n\t\t\t\"unix_priv_drop\",\n\t\t\t\"pprof_http\",\n\t\t\t\"proxy_haproxy_protocol\",\n\t\t\t\"network_hwaddr\",\n\t\t\t\"proxy_nat\",\n\t\t\t\"network_nat_order\",\n\t\t\t\"container_full\",\n\t\t\t\"candid_authentication\",\n\t\t\t\"backup_compression\",\n\t\t\t\"candid_config\",\n\t\t\t\"nvidia_runtime_config\",\n\t\t\t\"storage_api_volume_snapshots\",\n\t\t\t\"storage_unmapped\",\n\t\t\t\"projects\",\n\t\t\t\"candid_config_key\",\n\t\t\t\"network_vxlan_ttl\",\n\t\t\t\"container_incremental_copy\",\n\t\t\t\"usb_optional_vendorid\",\n\t\t\t\"snapshot_scheduling\",\n\t\t\t\"container_copy_project\",\n\t\t\t\"clustering_server_address\",\n\t\t\t\"clustering_image_replication\",\n\t\t\t\"container_protection_shift\",\n\t\t\t\"snapshot_expiry\",\n\t\t\t\"container_backup_override_pool\",\n\t\t\t\"snapshot_expiry_creation\",\n\t\t\t\"network_leases_location\",\n\t\t\t\"resources_cpu_socket\",\n\t\t\t\"resources_gpu\",\n\t\t\t\"resources_numa\",\n\t\t\t\"kernel_features\",\n\t\t\t\"id_map_current\",\n\t\t\t\"event_location\",\n\t\t\t\"storage_api_remote_volume_snapshots\",\n\t\t\t\"network_nat_address\",\n\t\t\t\"container_nic_routes\",\n\t\t\t\"rbac\",\n\t\t\t\"cluster_internal_copy\",\n\t\t\t\"seccomp_notify\",\n\t\t\t\"lxc_features\",\n\t\t\t\"container_nic_ipvlan\",\n\t\t\t\"network_vlan_sriov\",\n\t\t\t\"storage_cephfs\",\n\t\t\t\"container_nic_ipfilter\",\n\t\t\t\"resources_v2\",\n\t\t\t\"container_exec_user_group_cwd\",\n\t\t\t\"container_syscall_intercept\",\n\t\t\t\"container_disk_shift\",\n\t\t\t\"storage_shifted\",\n\t\t\t\"resources_infiniband\",\n\t\t\t\"daemon_storage\"\n\t\t],\n\t\t\"api_status\": \"stable\",\n\t\t\"api_version\": \"1.0\",\n\t\t\"auth\": \"trusted\",\n\t\t\"public\": false,\n\t\t\"auth_methods\": [\n\t\t\t\"tls\"\n\t\t],\n\t\t\"environment\": {\n\t\t\t\"addresses\": [],\n\t\t\t\"architectures\": [\n\t\t\t\t\"x86_64\",\n\t\t\t\t\"i686\"\n\t\t\t],\n\t\t\t\"certificate\": \"-----BEGIN CERTIFICATE-----\\nMIICCTCCAY+gAwIBAgIQLn+i9uv5Zs0Lx1twRZ0VNTAKBggqhkjOPQQDAzA2MRww\\nGgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRYwFAYDVQQDDA1yb290QGx4ZC10\\nZXN0MB4XDTE5MDkwNjE5MzMwOVoXDTI5MDkwMzE5MzMwOVowNjEcMBoGA1UEChMT\\nbGludXhjb250YWluZXJzLm9yZzEWMBQGA1UEAwwNcm9vdEBseGQtdGVzdDB2MBAG\\nByqGSM49AgEGBSuBBAAiA2IABDFJV9Oeg/YeuhVgtJbm0FtjpqtEWNOShstgk8U4\\nFO3Oaf+SjSAFcZ0YP89RQtr2bqNfUfH01vUzhi3yzfcg3WIgq0g0DMZbkXUEw8Bh\\nJOusaJKq/FXLPJTPi1jbZux1UaNiMGAwDgYDVR0PAQH/BAQDAgWgMBMGA1UdJQQM\\nMAoGCCsGAQUFBwMBMAwGA1UdEwEB/wQCMAAwKwYDVR0RBCQwIoIIbHhkLXRlc3SH\\nBJuKhsSHECABGfCwAQL7VAAC//5AOI0wCgYIKoZIzj0EAwMDaAAwZQIwBPrcDMhd\\naXxreL402xXOPIx8/QqWyu0GnGZIXh8B435hIpVgLKI6jzu8xHKUUm47AjEAmLxB\\nDgpGfBNg5xdjs0jPuqZ5lXRD3C9QJh6YzcBJpCJwOUfk7dVU4J4QdWTpz/r+\\n-----END CERTIFICATE-----\\n\",\n\t\t\t\"certificate_fingerprint\": \"d03a7cde11acb507305fd109b75b9c2628addd334a91c47fb849c7993e54e377\",\n\t\t\t\"driver\": \"lxc\",\n\t\t\t\"driver_version\": \"3.2.1\",\n\t\t\t\"kernel\": \"Linux\",\n\t\t\t\"kernel_architecture\": \"x86_64\",\n\t\t\t\"kernel_features\": {\n\t\t\t\t\"netnsid_getifaddrs\": \"false\",\n\t\t\t\t\"seccomp_listener\": \"false\",\n\t\t\t\t\"shiftfs\": \"false\",\n\t\t\t\t\"uevent_injection\": \"true\",\n\t\t\t\t\"unpriv_fscaps\": \"true\"\n\t\t\t},\n\t\t\t\"kernel_version\": \"4.19.71-0-vanilla\",\n\t\t\t\"lxc_features\": {\n\t\t\t\t\"mount_injection_file\": \"true\",\n\t\t\t\t\"network_gateway_device_route\": \"true\",\n\t\t\t\t\"network_ipvlan\": \"true\",\n\t\t\t\t\"network_l2proxy\": \"true\",\n\t\t\t\t\"network_phys_macvlan_mtu\": \"true\",\n\t\t\t\t\"seccomp_notify\": \"true\"\n\t\t\t},\n\t\t\t\"project\": \"default\",\n\t\t\t\"server\": \"lxd\",\n\t\t\t\"server_clustered\": false,\n\t\t\t\"server_name\": \"lxd-test\",\n\t\t\t\"server_pid\": 2333,\n\t\t\t\"server_version\": \"3.17\",\n\t\t\t\"storage\": \"dir\",\n\t\t\t\"storage_version\": \"1\"\n\t\t}\n\t}" t=2019-09-10T13:36:53-0400 lvl=dbug msg="Sending request to LXD" etag= method=GET url=http://unix.socket/1.0/containers/test t=2019-09-10T13:36:53-0400 lvl=dbug msg="Got response struct from LXD" t=2019-09-10T13:36:53-0400 lvl=dbug msg="\n\t{\n\t\t\"architecture\": \"x86_64\",\n\t\t\"config\": {\n\t\t\t\"image.architecture\": \"amd64\",\n\t\t\t\"image.description\": \"Alpine edge amd64 (20190906_13:00)\",\n\t\t\t\"image.os\": \"Alpine\",\n\t\t\t\"image.release\": \"edge\",\n\t\t\t\"image.serial\": \"20190906_13:00\",\n\t\t\t\"volatile.apply_template\": \"create\",\n\t\t\t\"volatile.base_image\": \"92d2c7c63c95dcfdb6fe0c34af6573bad4d76f23e328343ff52f45b5dca64721\",\n\t\t\t\"volatile.eth0.hwaddr\": \"00:16:3e:67:1d:11\",\n\t\t\t\"volatile.idmap.base\": \"0\",\n\t\t\t\"volatile.idmap.current\": \"[{\\\"Isuid\\\":true,\\\"Isgid\\\":false,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536},{\\\"Isuid\\\":false,\\\"Isgid\\\":true,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536}]\",\n\t\t\t\"volatile.idmap.next\": \"[{\\\"Isuid\\\":true,\\\"Isgid\\\":false,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536},{\\\"Isuid\\\":false,\\\"Isgid\\\":true,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536}]\",\n\t\t\t\"volatile.last_state.idmap\": \"[{\\\"Isuid\\\":true,\\\"Isgid\\\":false,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536},{\\\"Isuid\\\":false,\\\"Isgid\\\":true,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536}]\"\n\t\t},\n\t\t\"devices\": {},\n\t\t\"ephemeral\": false,\n\t\t\"profiles\": [\n\t\t\t\"default\"\n\t\t],\n\t\t\"stateful\": false,\n\t\t\"description\": \"\",\n\t\t\"created_at\": \"2019-09-06T15:35:11.409262822-04:00\",\n\t\t\"expanded_config\": {\n\t\t\t\"image.architecture\": \"amd64\",\n\t\t\t\"image.description\": \"Alpine edge amd64 (20190906_13:00)\",\n\t\t\t\"image.os\": \"Alpine\",\n\t\t\t\"image.release\": \"edge\",\n\t\t\t\"image.serial\": \"20190906_13:00\",\n\t\t\t\"volatile.apply_template\": \"create\",\n\t\t\t\"volatile.base_image\": \"92d2c7c63c95dcfdb6fe0c34af6573bad4d76f23e328343ff52f45b5dca64721\",\n\t\t\t\"volatile.eth0.hwaddr\": \"00:16:3e:67:1d:11\",\n\t\t\t\"volatile.idmap.base\": \"0\",\n\t\t\t\"volatile.idmap.current\": \"[{\\\"Isuid\\\":true,\\\"Isgid\\\":false,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536},{\\\"Isuid\\\":false,\\\"Isgid\\\":true,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536}]\",\n\t\t\t\"volatile.idmap.next\": \"[{\\\"Isuid\\\":true,\\\"Isgid\\\":false,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536},{\\\"Isuid\\\":false,\\\"Isgid\\\":true,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536}]\",\n\t\t\t\"volatile.last_state.idmap\": \"[{\\\"Isuid\\\":true,\\\"Isgid\\\":false,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536},{\\\"Isuid\\\":false,\\\"Isgid\\\":true,\\\"Hostid\\\":100000,\\\"Nsid\\\":0,\\\"Maprange\\\":65536}]\"\n\t\t},\n\t\t\"expanded_devices\": {\n\t\t\t\"eth0\": {\n\t\t\t\t\"name\": \"eth0\",\n\t\t\t\t\"nictype\": \"bridged\",\n\t\t\t\t\"parent\": \"lxdbr0\",\n\t\t\t\t\"type\": \"nic\"\n\t\t\t},\n\t\t\t\"root\": {\n\t\t\t\t\"path\": \"/\",\n\t\t\t\t\"pool\": \"default\",\n\t\t\t\t\"type\": \"disk\"\n\t\t\t}\n\t\t},\n\t\t\"name\": \"test\",\n\t\t\"status\": \"Stopped\",\n\t\t\"status_code\": 102,\n\t\t\"last_used_at\": \"1969-12-31T19:00:00-05:00\",\n\t\t\"location\": \"none\"\n\t}" t=2019-09-10T13:36:53-0400 lvl=dbug msg="Connected to the websocket" t=2019-09-10T13:36:53-0400 lvl=dbug msg="Sending request to LXD" etag= method=PUT url=http://unix.socket/1.0/containers/test/state t=2019-09-10T13:36:53-0400 lvl=dbug msg="\n\t{\n\t\t\"action\": \"start\",\n\t\t\"timeout\": 0,\n\t\t\"force\": false,\n\t\t\"stateful\": false\n\t}" t=2019-09-10T13:36:53-0400 lvl=dbug msg="Got operation from LXD" t=2019-09-10T13:36:53-0400 lvl=dbug msg="\n\t{\n\t\t\"id\": \"d46d8659-2ec3-424c-9352-5dcd19806d23\",\n\t\t\"class\": \"task\",\n\t\t\"description\": \"Starting container\",\n\t\t\"created_at\": \"2019-09-10T13:36:53.819339789-04:00\",\n\t\t\"updated_at\": \"2019-09-10T13:36:53.819339789-04:00\",\n\t\t\"status\": \"Running\",\n\t\t\"status_code\": 103,\n\t\t\"resources\": {\n\t\t\t\"containers\": [\n\t\t\t\t\"/1.0/containers/test\"\n\t\t\t]\n\t\t},\n\t\t\"metadata\": null,\n\t\t\"may_cancel\": false,\n\t\t\"err\": \"\",\n\t\t\"location\": \"none\"\n\t}" t=2019-09-10T13:36:53-0400 lvl=dbug msg="Sending request to LXD" etag= method=GET url=http://unix.socket/1.0/operations/d46d8659-2ec3-424c-9352-5dcd19806d23 t=2019-09-10T13:36:53-0400 lvl=dbug msg="Got response struct from LXD" t=2019-09-10T13:36:53-0400 lvl=dbug msg="\n\t{\n\t\t\"id\": \"d46d8659-2ec3-424c-9352-5dcd19806d23\",\n\t\t\"class\": \"task\",\n\t\t\"description\": \"Starting container\",\n\t\t\"created_at\": \"2019-09-10T13:36:53.819339789-04:00\",\n\t\t\"updated_at\": \"2019-09-10T13:36:53.819339789-04:00\",\n\t\t\"status\": \"Running\",\n\t\t\"status_code\": 103,\n\t\t\"resources\": {\n\t\t\t\"containers\": [\n\t\t\t\t\"/1.0/containers/test\"\n\t\t\t]\n\t\t},\n\t\t\"metadata\": null,\n\t\t\"may_cancel\": false,\n\t\t\"err\": \"\",\n\t\t\"location\": \"none\"\n\t}" Error: Common start logic: Failed to start device 'eth0': Failed to set the MAC address: Failed to run: ip link set dev veth1605a70f address 00:16:3e:67:1d:11: ip: ioctl 0x8933 failed: No such device Trylxc info --show-log test` for more info
 - [x] Output of the daemon with --debug (alternatively output of `lxc monitor` while reproducing the issue)
  ```


location: none
metadata:
  context: {}
  level: dbug
  message: 'New event listener: df543a70-d13e-4e50-beef-efc323875eab'
timestamp: "2019-09-10T13:38:15.41444506-04:00"
type: logging


location: none
metadata:
  context:
    ip: '@'
    method: GET
    url: /1.0
    user: ""
  level: dbug
  message: Handling
timestamp: "2019-09-10T13:38:19.311626444-04:00"
type: logging


location: none
metadata:
  context:
    ip: '@'
    method: GET
    url: /1.0/containers/test
    user: ""
  level: dbug
  message: Handling
timestamp: "2019-09-10T13:38:19.314156174-04:00"
type: logging


location: none
metadata:
  context:
    ip: '@'
    method: GET
    url: /1.0/events
    user: ""
  level: dbug
  message: Handling
timestamp: "2019-09-10T13:38:19.317748143-04:00"
type: logging


location: none
metadata:
  context: {}
  level: dbug
  message: 'New event listener: 5a66d16e-0f58-4519-81ad-a5cc7e70f75e'
timestamp: "2019-09-10T13:38:19.318676649-04:00"
type: logging


location: none
metadata:
  context:
    ip: '@'
    method: PUT
    url: /1.0/containers/test/state
    user: ""
  level: dbug
  message: Handling
timestamp: "2019-09-10T13:38:19.319409103-04:00"
type: logging


location: none
metadata:
  context: {}
  level: dbug
  message: 'New task operation: b5252085-ab23-45b5-9aa8-53b482a69ede'
timestamp: "2019-09-10T13:38:19.322220421-04:00"
type: logging


location: none
metadata:
  class: task
  created_at: "2019-09-10T13:38:19.320663789-04:00"
  description: Starting container
  err: ""
  id: b5252085-ab23-45b5-9aa8-53b482a69ede
  location: none
  may_cancel: false
  metadata: null
  resources:
    containers:
    - /1.0/containers/test
  status: Pending
  status_code: 105
  updated_at: "2019-09-10T13:38:19.320663789-04:00"
timestamp: "2019-09-10T13:38:19.322772416-04:00"
type: operation


location: none
metadata:
  context: {}
  level: dbug
  message: 'Started task operation: b5252085-ab23-45b5-9aa8-53b482a69ede'
timestamp: "2019-09-10T13:38:19.322796907-04:00"
type: logging


location: none
metadata:
  class: task
  created_at: "2019-09-10T13:38:19.320663789-04:00"
  description: Starting container
  err: ""
  id: b5252085-ab23-45b5-9aa8-53b482a69ede
  location: none
  may_cancel: false
  metadata: null
  resources:
    containers:
    - /1.0/containers/test
  status: Running
  status_code: 103
  updated_at: "2019-09-10T13:38:19.320663789-04:00"
timestamp: "2019-09-10T13:38:19.324894095-04:00"
type: operation


location: none
metadata:
  context:
    ip: '@'
    method: GET
    url: /1.0/operations/b5252085-ab23-45b5-9aa8-53b482a69ede
    user: ""
  level: dbug
  message: Handling
timestamp: "2019-09-10T13:38:19.327263261-04:00"
type: logging


location: none
metadata:
  class: task
  created_at: "2019-09-10T13:38:19.320663789-04:00"
  description: Starting container
  err: 'Common start logic: Failed to start device ''eth0'': Failed to set the MAC
    address: Failed to run: ip link set dev veth5c74bd45 address 00:16:3e:67:1d:11:
    ip: ioctl 0x8933 failed: No such device'
  id: b5252085-ab23-45b5-9aa8-53b482a69ede
  location: none
  may_cancel: false
  metadata: null
  resources:
    containers:
    - /1.0/containers/test
  status: Failure
  status_code: 400
  updated_at: "2019-09-10T13:38:19.320663789-04:00"
timestamp: "2019-09-10T13:38:19.342426558-04:00"
type: operation


location: none
metadata:
  context: {}
  level: dbug
  message: 'Failure for task operation: b5252085-ab23-45b5-9aa8-53b482a69ede: Common
    start logic: Failed to start device ''eth0'': Failed to set the MAC address: Failed
    to run: ip link set dev veth5c74bd45 address 00:16:3e:67:1d:11: ip: ioctl 0x8933
    failed: No such device'
timestamp: "2019-09-10T13:38:19.341981878-04:00"
type: logging
picture CosmicToast

All 8 comments

@5paceToast Hi, I've tried this on an Ubuntu 18.04 host and it works fine with an alpine image, so looks to be something specific to Alpine host.

Can you detail how you install LXD on alpine and I'll try to set it up and re-create the issue.

tomponline picture tomponline on 10 Sep 2019

@5paceToast can you also show the full output of ip a on the host before and after creating and trying to start the container. Thanks

tomponline picture tomponline on 10 Sep 2019

Hey @tomponline, no problem at all.
You install alpine, switch to the "edge" repositories and run apk add lxd.
To start the daemon you run rc-service lxd start.
Your /etc/apk/repositories file should look like so:

http://dl-cdn.alpinelinux.org/alpine/edge/main
http://dl-cdn.alpinelinux.org/alpine/edge/community
http://dl-cdn.alpinelinux.org/alpine/edge/testing

As for the outputs of ip a, before:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 56:00:02:40:38:8d brd ff:ff:ff:ff:ff:ff
    inet 155.138.134.196/23 brd 155.138.135.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2001:19f0:b001:2fb:5400:2ff:fe40:388d/64 scope global dynamic 
       valid_lft 2591680sec preferred_lft 604480sec
    inet6 fe80::5400:2ff:fe40:388d/64 scope link 
       valid_lft forever preferred_lft forever
3: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether 06:3a:5a:31:70:ff brd ff:ff:ff:ff:ff:ff
    inet 10.130.173.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 fd42:eb70:11df:bd4f::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::43a:5aff:fe31:70ff/64 scope link 
       valid_lft forever preferred_lft forever

After:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 56:00:02:40:38:8d brd ff:ff:ff:ff:ff:ff
    inet 155.138.134.196/23 brd 155.138.135.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2001:19f0:b001:2fb:5400:2ff:fe40:388d/64 scope global dynamic 
       valid_lft 2591666sec preferred_lft 604466sec
    inet6 fe80::5400:2ff:fe40:388d/64 scope link 
       valid_lft forever preferred_lft forever
3: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
    link/ether 06:3a:5a:31:70:ff brd ff:ff:ff:ff:ff:ff
    inet 10.130.173.1/24 scope global lxdbr0
       valid_lft forever preferred_lft forever
    inet6 fd42:eb70:11df:bd4f::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::43a:5aff:fe31:70ff/64 scope link 
       valid_lft forever preferred_lft forever
26: veth0@veth9da26f38: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 1e:cc:87:12:3c:88 brd ff:ff:ff:ff:ff:ff
27: veth9da26f38@veth0: <NO-CARRIER,BROADCAST,MULTICAST,UP,M-DOWN> mtu 1500 qdisc noqueue state LOWERLAYERDOWN qlen 1000
    link/ether 52:cc:18:2f:2f:72 brd ff:ff:ff:ff:ff:ff
CosmicToast picture CosmicToast on 10 Sep 2019
👍1

@5paceToast I've managed to reproduce this issue and it was as I suspected, LXD relies on the functionality of the ip command, specifically the ability to create a veth pair with both the host-side and the peer interface names specified (we don't guess what the OS will generate). However the ip command in busybox which comes by default with Alpine does not support specifying the peer name, only the host-side name. The solution is quite simple, apk add iproute2.

Are you able to ask the Alpine packagers to modify the the lxd package to require and install iproute2 by default?

As an aside, initially when starting the container I got this error:

lxc test 20190911081426.414 ERROR    cgroup - cgroups/cgroup.c:cgroup_init:54 - Failed to initialize cgroup driver

I managed to solve that by starting the lxc service first which mounted the cgroup filesystem, I wondered how you had LXD setup on your host to get the cgroup filesystem mounted?

tomponline picture tomponline on 11 Sep 2019

@stgraber maybe we should modify LXD to check for iproute2 installed, e.g.

ip -Version
ip utility, iproute2-ss190708

I've seen a similar issue before with LXC on Alpine as the ipvlan and router veth modes require the ip tool to setup proxy ARP and NDP entries, and the busybox tool doesn't support that either.

tomponline picture tomponline on 11 Sep 2019

@tomponline I think we need to put ip in neededPrograms so that we fail if it's entirely missing. However if someone installed a limited shim (like busybox), I don't think it's on us to go out of our ways to detect that.

stgraber picture stgraber on 11 Sep 2019
👍1

@tomponline re: cgroups, you can just start the cgroups service (same as with lxd).
You end up with a few missing things so my production deployment runs hand-written scripts that I never managed to get in upstream.

Confirming that with iproute2 installed everything works as expected.
Just notified the alpine maintainer that LXD should depend on iproute2, thanks!

CosmicToast picture CosmicToast on 11 Sep 2019
👍1

thanks @5paceToast

tomponline picture tomponline on 11 Sep 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

fwaggle picture fwaggle  路  4Comments
spacekookie picture spacekookie  路  3Comments
kp3nguinz picture kp3nguinz  路  5Comments
iteco picture iteco  路  3Comments
chuegel picture chuegel  路  3Comments