Singularity: --contain option still mounts over /home

[remram44]

cc ViDA-NYU/reprozip#267

I am trying to run a container that would let me see all the files from the image, mounting nothing from the host. From reading the help, it seems that the --contain option does just that:

    -c/--contain        This option disables the sharing of filesystems on
                        your host (e.g. /dev, $HOME and /tmp).

However using --contain the /home directory is replaced by a blank one, with only a single folder matching my user name on the host, also empty. I cannot see any file from either the image or $HOME on the host.

To achieve what I want it is necessary to move $HOME out of the way using the --home option, e.g.:

% singularity shell -c -H $(pwd):/otherhome myimage.tar.gz
/otherhome$ ls /home
remram
fchirigati

I reported on Slack that this was surprising and kind of a weird workaround and @GodloveD told me to open this issue.

Log:

/home/remram% ls
myimage.tar.gz

/home/remram% singularity shell myimage.tar.gz
Opening gzip compressed archive, stand by...
Singularity: Invoking an interactive shell within container...
/home/remram$ ls
myimage.tar.gz
/home/remram$ ls /home
remram
vagrant
/home/remram$ exit

/home/remram% singularity shell -c myimage.tar.gz
Opening gzip compressed archive, stand by...
Singularity: Invoking an interactive shell within container...
/home/remram$ ls
/home/remram$ ls /home
remram
/home/remram$ exit

/home/remram% singularity shell -c -H $(pwd):/otherhome myimage.tar.gz
Opening gzip compressed archive, stand by...
Singularity: Invoking an interactive shell within container...
/otherhome$ ls
myimage.tar.gz
/otherhome$ ls /home
vagrant
/otherhome$ exit

[DrDaveD]

@remram44 What would you propose that singularity --contain should set the default $HOME and pwd to inside the image? Since it always runs as the original user, I don't think it will be very common for someone to want home directories of multiple users inside of an image. Also, by keeping the home directory outside of the image, the image can be used by any number of users and does not have to be customized for each user.

[remram44]

I don't think it's very common for --contain to be required either, but I think when it is, it should do something more useful. Even if setting it you end up with $HOME not existing, that is easy enough to fix (doing HOME=... or using your -H option).

Alternatively you could fix the help message or remove the option (please don't though, I need it).

[remram44]

To answer the question, the behavior I expect is to not have Singularity do any mount for home at all.

[DrDaveD]

I agree that an empty home directory is not very useful. However, $HOME does have to be set to something because the shell depends on it, so what would it be set to by default? Most images will not have a /home/$LOGNAME in them so singularity can not count on that.

The help message of --contain is not incorrect, it does not share $HOME from the host, so how could it be improved to describe what it is doing? It is also making an empty /tmp.

I think most users of --contain are going to want to also use -H/--home unless they don't care about the home directory. In your case I recommend keeping the home directories outside of the image.

[remram44]

The help message of --contain is not incorrect, it does not share $HOME from the host, so how could it be improved to describe what it is doing? It is also making an empty /tmp.

It is not incorrect in that it doesn't share those filesystems, however it does create a mount from an empty folder instead, which we have no way of inferring from the help string. If you do something new, I recommend mentioning it explicitly.

    -c/--contain        Mount empty directories over /dev, $HOME and /tmp
                        instead of sharing those filesystems from your host.

We can argue that accessing the /home directory from the image is not the usual need, but that's why it is behind an option. Some people will build images of their whole environment, and there should be a way (with an option) to have those images work.

Right now, I need to use -H with a non-existent target folder (/otherhome) to have Singularity not hide my files, then change back the HOME variable in the shell and load my rc file again. That is a lot of work for something so obvious (albeit a little bit unusual).

[DrDaveD]

That's a pretty good help message, except that /dev isn't empty by default, it is minimal. I submitted pr #885 to change the help message with a variation on your suggestion.

[remram44]

I will therefore have to use -H with some auto-generated target name so it doesn't clash. This is unfortunate for ReproZip integration.

[gmkurtzer]

We can change some of the handling of this, but for almost every application, a writable home directory is required. The idea of contain, is that it allows for an application to be executed within a container in a 'contained' manner such that it doesn't touch the host's real file systems. If we don't create a writable home directory location, many applications would error.

Instead of changing existing behavior, I would recommend that we add an option to do what you need. Perhaps something like --nomounts would be sufficient?

[dtrudg]

Closing. There has been no further discussion for 2 years, and contain vs containall is explained in the 3.x help.

[remram44]

--containall is NOT a fix for the issue, as described above it still mounts over the HOME directory (it just mounts an empty dir instead of host HOME).

Similarly, --no-home mounts whatever CWD into the image.

I'm trying not to mount anything in, I'm surprised Singularity always has to be given something to mount somewhere.

I am happy to discuss this further, you don't have to close it on me like this.

[dtrudg]

Hello @remram44 - apologies - given no update for 2 years I did a quick review, and verified the 3.x message, and --no-home make sense.

You should be able to do e.g.:

cd /tmp 
$ singularity run --contain --no-home docker://ubuntu

And that will drop you in the container at / without a home dir mounted in.

[remram44]

Hi @dctrud,

That's great news, thanks!

The doc for --no-home mentions:

mount your current working directory (CWD) without mounting your host $HOME directory with the --no-home flag

So your example is mounting the host's /tmp to /tmp in the image, correct?

[dtrudg]

Actually that cd /tmp is not needed... you can be in your home dir when you do it. I've built a container and put /home/in_the_container as a file in the container. Then...

dave@piran~> pwd
/home/dave
dave@piran~> singularity run --contain --no-home test.sif
Singularity> ls /home
in_the_container

[remram44]

This seems to serve my purpose however it is inconsistent with the note in the docs which state:

Beware that if it is the case that your CWD is your $HOME directory, it will still mount your $HOME directory.

Might be just a case of the doc being slightly out of date?

[dtrudg]

The doc is sort of accurate. If you use --no-home alone you'll get your own $HOME if it's CWD. Using --no-home with --contain together stops that.

I'll have a think about how to clarify the doc.

[remram44]

I see, that makes sense. Thanks again. I will be able to get rid of my -H hack.

[jmstover]

I'll speak up here quickly since I wrote the initial --no-home option. Basically, it subverted the automatic mounting of the users $HOME, but I didn't want to mess with the CWD mount logic.

If you're in the directory: /home/user/mydir1 ... and use the --no-home option, then that directory will be bind mounted as it's your working directory. But the rest of your $HOME will not be availible.

Meaning if you do:

cd ~/
singularity shell --no-home lolcow.sif

You will have your full home in the container, even with the --no-home option, as it's your current working directory.

The option was added in for the use case where, by default, users homes were wanted to be bind mounted in, but there were some cases where that was not wanted and --contain couldn't be used. So, the following would be done instead:

cd /tmp
singularity shell --no-home lolcow.sif

[vmiheer]

I am trying this but seems like it is mounting home anyway. Which version of singularity it's fixed?

vmiheer@kp36123 ➜  /tmp pwd
/tmp
vmiheer@kp36123 ➜  /tmp singularity --version
singularity version 3.6.4
vmiheer@kp36123 ➜  /tmp singularity run --contain --no-home ~/singularity/notchpeak.sif
Singularity> ls -al
total 2144
drwxr-xr-x  74 vmiheer mygroup   8192 Nov  7 18:27 .
drwxr-xr-x 309 root     root            0 Nov  7 13:40 ..
drwxr-xr-x  17 vmiheer mygroup   4096 Nov 11  2019 al-folio
-rw-r--r--   1 vmiheer mygroup     29 Aug 23  2019 .aliases
-rw-------   1 vmiheer mygroup  81376 Nov  7 18:25 .bash_history
-rw-r--r--   1 vmiheer mygroup     24 Oct 29  2015 .bash_logout
-rw-r--r--   1 vmiheer mygroup    191 Oct 22 13:54 .bash_profile
-rw-r--r--   1 vmiheer mygroup   2920 Oct 22 13:48 .bashrc
-rw-rw----   1 vmiheer mygroup    348 Oct 13 19:48 bc.log
drwxr-xr-x   2 vmiheer mygroup    138 Oct 13 19:44 bin
drwxr-xr-x   3 vmiheer mygroup     27 Nov  7  2019 .bundle
drwxr-xr-x  22 vmiheer mygroup   4096 Sep 21 22:17 .cache
drwxr-xr-x  19 vmiheer mygroup    256 Oct 26 11:35 .ccache
drwxr-xr-x   3 vmiheer mygroup     58 Oct  2 00:34 .cgdb

[dtrudg]

@vmiheer - this does not replicate for me with 3.6.4

dave@dev-fedora/tmp> pwd
/tmp
dave@dev-fedora/tmp> singularity version
3.6.4
dave@dev-fedora/tmp> singularity run --contain --no-home ~/alpine_latest.sif
Singularity> pwd
/
Singularity> ls /home/dave
ls: /home/dave: No such file or directory
Singularity> 

It's possible Singularity has been configured to mount your home directory, or a parent of it, in a different route. Where is your $HOME on the system?

To diagnose the issue further, please let us know what configuration options have been set in singularity.conf on your system, and include debug output for your singularity run command with the -d flag - singularity -d run ...