Lxd: Mounting directories r/w

[techtonik]

592 describes the way (actual at lest for 0.10) to mount directories read only:

lxc config device add contname sharename disk source=$PWD path=/home/pathinside

How to mount them with write privileges?

---

This command above mounts the folder nobody:nobody, and it is impossible to create anything in it.

@stgraber mentions that this command supports three types of mounts:

1. partitions - `source=
2. block devices - source=/dev/blockdevice
3. file or dir - source=/host/file

The actual for this issue (3) is mounted using "bind mount", and it looks like LXC requires some kind of access control like ownership to enable r/w for that. So, LXD needs to map directory owner from host to guest. So, how to do this?

And also, from the user point of view that mapping is not always necessary. I'd be happy to operate with directory that has write and execution permissions for everybody. Is it possible to teach LXD/LXC to share directory in this way?

Or it is possible to make automatic mapping from current user to root? Like:

lxc config device add contname sharename disk source=$PWD path=/home/pathinside rwmap=$USER:root

?

Ideally, there should be something that is easy to memorize, and what could have dedicated help page.

lxd mount host=$PWD guest=/root/somename

[techtonik]

So, can anybody tell me how to make a shared folder writable with LXD? =(

[stgraber]

That's actually harder than it may seem and short of having some kind of uid/gid switching in-kernel filesystem, there's no single good solution for LXD to do that.

For now, what you need to do is setup a bind-mount as explained before, then make the folder on the host writable by the uid/gid in the container. You can do so with:

• Changing the file/dir mode to allow writes by all users/groups
• Chowning the file/dir to the uid/gid in the container
• Setting up POSIX ACLs, allowing the uid/gid in the container access to the folder.

[techtonik]

@stgraber thanks for clarification. How can I get UID/GID of container programatically for my chmod command? (I need to get root user)

[stgraber]

The uid/gid will be the base allocation for the lxd user (can be found in /etc/subuid and /etc/subgid) + the uid/gid inside the container.

So here the "lxd" user has "lxd:100000:65536" which puts the first uid at 100000, so root will be 100000 and uid 100 would be 100100.

As for setting the ACLs, you can do that with setfacl, using those uid/gids.

[techtonik]

So here the "lxd" user has "lxd:100000:65536" which puts the first uid at 100000, so root will be 100000 and uid 100 would be 100100.

This is only true if you have one container running, but what if you have several? Is there a way to query running container about its uid/gid mappings?

[stgraber]

We do not currently do per-container uid/gid maps, all containers run by LXD share the same map, so what I said above will work fine.

[benaryorg]

I have found a slightly different solution by just setting the uid and gid flag when bind-mounting. It does not change the hosts permissions.

My current entry in lxd looks like this:

raw.lxc: lxc.mount.entry = <src> <dest> none bind,create=dir,rw,uid=100000,gid=100000 0 0

as there is no native support in lxd to set these flags as far as I have seen.

Okay looks as if it does not work yet.

[stgraber]

what filesystem are you bind-mounting? I can see how this may work with vfat but I'm not sure how a uid/gid aware filesystem would work.

[benaryorg]

I tried ext4, although it did not work.

I could also try btrfs if you'd like to hear the results.

[pcdummy]

Both ext4 and btrfs are uid/gid aware. So both of them don't work without ACL's.

This is my setfacl line which i use for a container data directory:

setfacl -Rm user:lxd:rwx,default:user:lxd:rwx,user:100000:rwx,user:101000:rwx,default:user:100000:rwx,default:user:101000:rwx /cdata/saltmaster

It gives the container root (100000) and the default container user (101000) permanent access to this directory.

[benaryorg]

Is there a way to ensure that any newly created files (from outside the container) in the shared directory inherit the ACLs?

[stgraber]

That's what the "default:" keyword does, or at least that's my (limited) understanding of POSIX ACLs.

[benaryorg]

Oh. Okay, that would have been useful to know one or two days ago.

Thank you for that.

[jasonblalock]

I'm trying to follow the instructions here: https://github.com/lxc/lxd/issues/592#issuecomment-99904045

I created a group lxd-share with gid 1001 and added my user on the host machine. chowned the directory to that group.

host /etc/subgid

jason:100000:65536
lxd:165536:65536
root:165536:65536
root:1001:1

id_map

name: trusty-01
profiles:
- default
config:
  raw.lxc: |
    lxc.id_map = g 400000 1001 1

guest /etc/group: http://paste.ubuntu.com/13071847/

Calling id inside guest: uid=0(root) gid=0(root) groups=0(root)
Calling sudo id inside guest: uid=0(root) gid=0(root) groups=0(root),400000(lxd-share)

So I can successfully edit the shared files if I do it as sudo, but I can't edit the files without sudo. How do I get it to work without sudo?

[hallyn]

Add lxd-share (400000) to your user's list of groups in the container (using /etc/group) so that 'id' inside the guest shows lxd-share.

[jasonblalock]

Maybe I'm misunderstanding your instructions, but I've already added lxd-share:x:400000:root to my /etc/group file inside the container as seen here: http://paste.ubuntu.com/13071847/.

When I /bin/bash into a clean container my user is root. But using lxd-share:x:400000:root doesn't add the group to root as indicated by id. It only adds the group to sudo for some reason hence why sudo id shows the lxd-share group.

[hallyn]

Oh, sorry, I though the first 'id' in your previous output was as the user. It's as root.

I'm actually not clear why the first shell isn't enabling your supplementary groups.

[jasonblalock]

No problem. When I create a standard container and enter it via lxc exec trusty-01 /bin/bash, the default user is root correct?

Also continuing to look into this, I'm seeing some very odd behavior.

# starting as the host
jason@ubuntu:~$ lxc exec trusty-01 /bin/bash
root@trusty-01:~# id
uid=0(root) gid=0(root) groups=0(root)
root@trusty-01:~# id root
uid=0(root) gid=0(root) groups=0(root),400000(lxd-share)
root@trusty-01:~# sudo id
uid=0(root) gid=0(root) groups=0(root),400000(lxd-share)

Somehow this is the cause of my issue, but I don't know how? id root clearly shows that it has lxd-share, yet, id doesn't.

[stgraber]

That'd be because LXC doesn't setup your groups on attach.

Unfortunately doing so requires knowledge of the distribution inside the container which LXC doesn't have, so I expect it's just not bothering to do nss initialization (init_groups()).

We could probably update LXC to attempt nss initialization using the same location as the host it's running on (so using the nss functions of glibc) and if that fails, then continue without the group initialization.

@hallyn thoughts?

[hallyn]

No, I must have been having quite the brain fart yesterday. Obviously just running bash won't call the pam session plugin, which is the one which does this.

Manually reaching into the container to poke nss seems like begging for trouble or container security breaches.

Doing

    lxc exec container -- su - root

gets you the list of groups. So if we cared, the simplest thing would be to have a simple alias for that, or something like

lxc login container [@user - root by default ]

which is hard to keep semantically clean.

[jasonblalock]

So if I understand this correctly by calling su - root we "switch" to root even though it's already the default, but by doing so it triggers the group initialization?

Aside from triggering the group initialization, is it any different than /bin/bash? All I'm trying to do is use lxd/lxc for creating development environments that don't pollute my host, but I want to be able to edit files outside the container that are accessed inside. If I start installing things after using su - root everything will run the same as if I had entered /bin/bash the difference is the supplementary groups will be there?

And from a security perspective is the lxc exec container -- su - root ok to use? Thanks for all your help.

[hallyn]

On Thu, Nov 05, 2015 at 05:10:12PM -0800, Jason Blalock wrote:

So if I understand this correctly by calling su - root we "switch" to root even though it's already the default, but by doing so it triggers the group initialization?

Right, the 'su' program in the container itself will call its PAM plugin
to setup groups. Then run the user's default shell or the specified command.

Aside from triggering the group initialization, is it any different than /bin/bash?

Root may well have another shell as its default shell, i.e. dash. So
you may need to tell su to run bash if that's what you need, i.e.
su - root -c bash

[hallyn]

And from a security perspective is the lxc exec container -- su - root ok to use? Thanks for all your help.

(sorry skipped htis in last reply) - yes, it is ok to use afaik.

[srkunze]

That'd be because LXC doesn't setup your groups on attach.

It may be just me but I have to admit I would expect LXC doing this for me.

Unfortunately doing so requires knowledge of the distribution inside the container which LXC doesn't have, so I expect it's just not bothering to do nss initialization (init_groups()).

That sounds like another hook to me. Each image/container should tell the hypervisor where to find the triggers for group initialization.

(Disclaimer: I am honestly surprised that group initialization is some sort of a separate/optional module; when I consider all the scripts/modules I know which simply assume proper setup of all additional groups...)

[srkunze]

@stgraber

We do not currently do per-container uid/gid maps, all containers run by LXD share the same map, so what I said above will work fine.

Can this be abused (aka security hole)?

[stgraber]

All containers sharing the same map, makes resource starvation DoS possible, yes. That is, anything covered by ulimits will be counted together for all containers using the same uid/gid.

[stgraber]

As for initializing the group list, the problem is that this is a purely userspace thing, Android does it differently from other distros and even within distros things may vary widely.

Having LXC or LXD parse a file from inside the container and then call a bunch of syscalls from outside the container is a security risk and one we've been unwilling to take for LXD so far (that's also why we don't allow script/binary hooks inside LXD).

[srkunze]

Having LXC or LXD parse a file from inside the container

I thought hooks/templates are not part of the container but a mere description (residing besides the container) of where to find things inside.

call a bunch of syscalls from outside the container

How does "su - root" do it then? Correct me if I am wrong but this cmd runs inside the container and changes things inside.

[stgraber]

So there are a few things:

• We don't ever want to parse a container file from outside the container as if we screw up the parser, you'd be running code as root on the host. That concern is about parsing /etc/passwd and /etc/group specifically, not about parsing container metadata (which can't be modified from inside the container).
• The location and syntax of the files to parse varies between Linux distributions.
• Anyone using libnss plugins in /etc/nsswitch.conf for passwd or group wouldn't get the right groups as those plugin need to be loaded and executed and may then depend on network connection.

su -u root runs entirely inside the container through the LXC attach interface, so hits the container's C library which may then talk to libnss which will then parse /etc/nsswitch.conf and call the relevant plugins.

We could extend the attach interface in liblxc, though that may well be an ABI break so we'd need a new library symbol for it... Such an extension would have liblxc make a good guess and parse /etc/passwd, /etc/group and /system/etc/passwd and /system/etc/group from inside the container so avoiding the big security issue. That still wouldn't work for anyone using libnss-extrausers, sssd or any other nss plugin though.

[srkunze]

@stgraber Thanks for you detailed response. I am not so deep into that matter, however, I feel that all tools which do the parsing already (like "su -") are available in the container. Otherwise, the container does not support proper group setup.

Thus, why not having a "pre-attach" hook that basically call a predefined cmd (available in the container) to setup the group initialization (such as "su-")?

So, no extension that guesses and parses things (which I generally dislike).

[stgraber]

You may be able to do something with the lxc command aliases, basically aliasing "lxc exec container COMMAND" to "lxc exec CONTAINER su - COMMAND".

@tych0 does the aliasing stuff allow for re-organizing arguments like that? ^

[tych0]

On Tue, Dec 01, 2015 at 09:19:23AM -0800, Stéphane Graber wrote:

You may be able to do something with the lxc command aliases, basically aliasing "lxc exec container COMMAND" to "lxc exec CONTAINER su - COMMAND".

@tych0 does the aliasing stuff allow for re-organizing arguments like that? ^

I think so, something like:

aliases:
  command: su - command

should work.

---

Reply to this email directly or view it on GitHub:
https://github.com/lxc/lxd/issues/714#issuecomment-161038137

[srkunze]

So, this needs to be part of the container's metadata, right?

[stgraber]

Seems like a lot of the things discussed in there have been addressed through fixes or workarounds, the rest is mostly design decision, closing this catch all issue.

If there are specifics that still need addressing, please file specific issues.

[naisanza]

@stgraber how does it handle multiple containers mounting the same path, does it allow it?