Singularity: ability to mount containers as non-root writable should be configurable

[GodloveD]

Using the Ubuntu definition file in the examples directory, these commands:

dave@trash:~$ sudo singularity create ubuntu.img
dave@trash:~$ sudo singularity bootstrap ubuntu.img singularity/examples/ubuntu.def
dave@trash:~$ sudo singularity shell -w ubuntu.img
Singularity.ubuntu.img> # mkdir /foo
Singularity.ubuntu.img> # exit
dave@trash:~$ sudo chown dave:dave ubuntu.img
dave@trash:~$ singularity shell -w ubuntu.img
Singularity.ubuntu.img> $ touch /foo/bar

yield output:

touch: cannot touch 'bar': Permission denied

That makes sense.

But these commands yield output that may be surprising:

dave@trash:~$ id
uid=1001(dave) gid=1001(dave) groups=1001(dave),4(adm),27(sudo),30(dip),44(video),46(plugdev),110(lxd)
dave@trash:~$ sudo singularity shell -w ubuntu.img
Singularity.ubuntu.img> # useradd -u 1001 dave
Singularity.ubuntu.img> # chown dave:dave /foo
Singularity.ubuntu.img> # exit
dave@trash:~$ singularity shell -w ubuntu.img
Singularity.ubuntu.img> $ touch /foo/bar
Singularity.ubuntu.img> $ ls -l /foo/bar

output is (drumroll...)

-rw-rw-r-- 1 dave dave 0 Dec 23 22:16 /foo/bar

This behavior should be configurable to avoid ambiguity.

[serious-steve]

@GodloveD not sure if I understand what you want to demonstrate here.
In the second snipped you chowned the /foo dir whereas in the first snippet it is still root owned.
What exactly surprises you about the second snippet? I can write the image by just running this:

sudo singularity create ubuntu.img
sudo chown 1000:1000 ubuntu.img
sudo singularity bootstrap ubuntu.img Work/singularity/examples/ubuntu.def
sudo singularity shell -w ubuntu.img
Singularity: Invoking an interactive shell within container...

Singularity.ubuntu.img> # mkdir /foo && chown 1000:1000 /foo
Singularity.ubuntu.img> # exit

id -g
1000

singularity exec -w ubuntu.img touch /foo/bar

Is it you want to have an option to disable non-root write access to that image altogether?
Then you just need to omit the sudo chown 1000:1000 ubuntu.img

[GodloveD]

@c1t4r I guess it's only surprising if you read the docs lol! There it says "... if you want to make changes to your container, you must be root inside your container, which means you must first become root outside your container. " Obviously this example shows that it's not strictly true. But I think this case is a loophole and the intended behavior is that only root can write to a container.

This feature/bug seems undesirable because it breaks portability. Your example above works because you have the same UID on the build system and the deployment system. But what if you copy the container to a new system and try to mount it as writable? Unless your UIDs are identical on both systems, your container is now busted. That is why I went through the extended example of creating a user within the container with a certain UID. You could match your UID on the deployment system with this new user and force it to work in that one instance, but is that a feature or a bug?

This issue came up because one of our HPC users did exactly this and then shot themselves in the foot by mounting their image as writable on multiple nodes and corrupting it. When I explained the situation on Slack, @gmkurtzer suggested that I raise an issue. So, here it is.

What I want is an option (probably in singularity.conf) to disable non-root users from mounting containers as writable even on images they own. That way things are not ambiguous.

[serious-steve]

@GodloveD thanks for your explanation. I understand better now. Indeed I was surprised myself to learn that I could shell into an owned container file and write into it like that. I wasn't aware of that being even possible. I recall the docs exactly as you cited them - that you need to be root.

I can also understand the problematic side effects of non-matching UIDs/GIDs.
On the other hand I like the possibility to have exec -w / shell -w.
Since you pledge for an config option to disable it I assume this functionality was implemented intentionally and not documented properly yet.
In that case the documentation should be fixed, too, and we possibly should come up with a nice example of exec -w for a users container.

[gmkurtzer]

I am not sure how or rather if we want to fix this. Perhaps we just want to align the documentation to what is actually happening. If we limit the ability to mount a container as writable even if the image file itself is owned and/or writable by the calling user, then we no longer will be able to do things like 'import' as non-root which is a feature of the 'development' branch.

Making this configurable means there is no standard behavior (but there is default).

Thoughts?

[GodloveD]

I really just wanted to bring this to your attention and see if you thought
it was a feature or a bug. If it's a feature (sounds like it is) I agree
that docs need updated.

On Mar 25, 2017 3:05 PM, "Gregory M. Kurtzer" [e-mail]
wrote:

I am not sure how or rather if we want to fix this. Perhaps we just want
to align the documentation to what is actually happening. If we limit the
ability to mount a container as writable even if the image file itself is
owned and/or writable by the calling user, then we no longer will be able
to do things like 'import' as non-root which is a feature of the
'development' branch.

Making this configurable means there is no standard behavior (but there is
default).

Thoughts?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/singularityware/singularity/issues/427#issuecomment-289232485,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHUUXAatcr2kYslbTOjj3FJ98CCq_yLCks5rpWWBgaJpZM4LVF78
.

[vsoch]

I started this today! @GodloveD could you dump the summary of what we learned in this thread (that should be in the docs) and I'll make it happen? These will be released with 2.3.

[GodloveD]

@vsoch woo hoo!

So, right now the docs here say

if you want to make changes to your container, you must be root inside your container, which means you must first become root outside your container. Additionally you will need to tell Singularity that you wish to mount the container as --writable so you can change the contents.

But in light of this this feature that's not really true right? I guess they should just say something more along the lines of...

if you want to make changes to your container, you must mount the container as --writable so you can change the contents. Note that standard Linux ownership and permission rules pertain to files within the container, so the --writable option does not guarantee you can do things like install new software. This might be a bit confusing if you copy a container from one computer to another. If your pids are different on the two computers you will lose the ability to edit files you previous had write access to. In these instances, it might be best to modify your container as root and so you would first need to become root outside of the container.

Or something (that needs a lot of editing).

If you want to make a 2.3 branch in https://github.com/singularityware/singularityware.github.io I would be happy to help you work on the docs. I understand if you want to put the first draft out yourself though too.