Lxd: /proc/sys/net/bridge folder missing inside of container, even with kernel modules enabled

Created on 10 May 2017 · 10Comments · Source: lxc/lxd

Required information

Distribution: Ubuntu 17.04 x64

$ lxc info
config:
  core.https_address: '[::]'
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
api_status: stable
api_version: "1.0"
auth: trusted
public: false
environment:
  addresses:
  - 192.168.1.231:8443
  - 192.168.122.1:8443
  - 10.201.128.1:8443
  - 10.99.0.1:8443
  - 172.17.0.1:8443
  - 172.18.0.1:8443
  architectures:
  - x86_64
  - i686
  driver: lxc
  driver_version: 2.0.7
  kernel: Linux
  kernel_architecture: x86_64
  kernel_version: 4.10.0-20-generic
  server: lxd
  server_pid: 3348
  server_version: "2.12"
  storage: zfs | dir
  storage_version: 0.6.5.9-2 | 1

Issue description

/proc/sys/net/bridge folder missing inside of container, even with kernel modules enabled

Steps to reproduce

I am trying to run Docker Swarm inside of LXD. Docker swarm needs bridged networking, and is specifically looking for files in /proc/sys/net/bridge

I have included the br_netfilter kernel module in my profile, I thought that was supposed to enable the /proc/sys/net/bridge folder.

My host has content in /proc/sys/net/bridge:

host $ ls /proc/sys/net/bridge/
bridge-nf-call-arptables  bridge-nf-call-ip6tables  bridge-nf-call-iptables  bridge-nf-filter-pppoe-tagged  bridge-nf-filter-vlan-tagged  bridge-nf-pass-vlan-input-dev

My container does not have a /proc/sys/net/bridge folder though:

container # ls /proc/sys/net
core  ipv4  ipv6  netfilter  unix

Is there some step I'm missing to get /proc/sys/net/bridge to appear? I tried mounting it in, but then the container would not start.

Information to attach

LXC Profile:

config:
  boot.autostart: "true"
  linux.kernel_modules: bridge,br_netfilter,ip_tables,ip6_tables,netlink_diag,nf_nat,overlay
  raw.lxc: |-
    lxc.aa_profile = unconfined
    lxc.cgroup.devices.allow = a
    lxc.mount.auto=proc:rw sys:rw
    lxc.cap.drop =
  security.nesting: "true"
  security.privileged: "true"
devices:
  eth0:
    nictype: bridged
    parent: lxdbr0
    type: nic
  root:
    path: /
    pool: default
    type: disk

Source

caleblloyd

👍4

Most helpful comment

You have to touch /.dockerenv to tell Docker it is running in a containerized environment, then it will skip the check for that file.

caleblloyd on 19 Oct 2017

👎4 👍3

All 10 comments

Sorry, I wrote a reply to this a few days ago but forgot to hit send :)

I can confirm that /proc/sys/net/bridge doesn't show up in privileged or unprivileged containers. The reason is most likely that br_netfilter isn't network namespace aware and so only shows up in the host.

That's a kernel bug (or limitation) and not something that LXD itself can do anything about.

You'll need a kernel developer to look into the br_netfilter code, check if it's network namespace aware (should be since netfilter is) and then modify the proc handling code to also have those entries show up in containers.

stgraber on 24 May 2017

@stgraber , what would be a possible workaround to successfully run kubeadm init ? (asked here https://askubuntu.com/questions/903344/lxd-getting-proc-sys-net-bridge-bridge-nf-call-iptables)
There are successful k8s installations inside lxd with conjure-up https://insights.ubuntu.com/2017/02/20/running-kubernetes-inside-lxd/

hutm on 24 Jul 2017

@hutm did you try using the LXD profile at the top of this issue? I've gotten it to work with the most recent versions of Docker Swarm and OpenShift Origin (includes Kubernetes)

caleblloyd on 24 Jul 2017

@caleblloyd i'm not able to get it working - loaded br_netfilter on the lxc host, copied your config, upgraded to kernel 4.10.0-37...Still, the directory does not exist on in the lxc container.

root@container:~# lsmod | grep br_netfilter
br_netfilter           24576  0
bridge                139264  1 br_netfilter
root@contianer:~# ls -l /proc/sys/net/
total 0
dr-xr-xr-x 1 root root 0 Oct 19 15:08 core
dr-xr-xr-x 1 root root 0 Oct 19 15:08 ipv4
dr-xr-xr-x 1 root root 0 Oct 19 15:08 ipv6
dr-xr-xr-x 1 root root 0 Oct 19 15:09 netfilter
dr-xr-xr-x 1 root root 0 Oct 19 15:08 unix

Did you change anything?
I'm also trying to run kubeadm init (which fails because of that).
@hutm did you get it to work?

Thanks