Singularity: [idea] Make passing environment variables and mounting $HOME optional and off by default

Created on 4 Jan 2017  路  23Comments  路  Source: hpcng/singularity

The main advantage of using singularity is providing reproducible software environment that is independent of the host machine. However, many new users of singularity stumble into problems with host environment being mixed up with the software inside the container. This is caused by two issues:

  • Environment variables available on the host are passed to the container image. For example, without taking care to unset PYTHONPATH this can lead to software from the container using libraries from the host.
  • By default, singularity mounts the $HOME folder. This can lead to some dot files (for example .Renviron) being read and in turn make software from inside of the container use libraries from the host.

Both of those features can be useful if user want to mix software from the host with software inside the container, but they cause a lot of troubles to new users. Therefore I would propose making them optional (from a perspective of a non-root end user - in a form of command line flag) and disable them by default.

picture chrisgorgo
👍9

All 23 comments

+1. The passing of env variables has made debugging of containers on TACC quite challenging for me.

poldrack picture poldrack on 4 Jan 2017

+1 again @chrisfilo - I think we could think of a good way to include this in our refactor to add an %environment section to the spec and .env folder in the image itself, with various environments that the user can choose (or not choose) to include. There has been a lot of work/discussion on this, most recent discussion here and likely it's just a matter of getting caught up after break. @bauerm97 @bbockelm @gmkurtzer looping you into this.

vsoch picture vsoch on 4 Jan 2017

I think this should be handled by a command line flag rather than an image description file so users using images converted directly from Docke Hub could benefit from it.

chrisgorgo picture chrisgorgo on 4 Jan 2017
👍1

yes the underlying functionality would be accessible in multiple ways, only one of them being the spec file.

vsoch picture vsoch on 4 Jan 2017
👍1

Without thinking too deeply on the proposal itself --

I think it's a bit dangerous to break backward compatibility for existing sites / containers. While this is a trip hazard for new users, it's worth noting that _existing_ users have already tripped, fallen, and written a workaround to get back up. Let's not break the workaround!

What _does_ make sense to me is:

  • Newly created images can specify how they want environment variables handled.

    • There are lots of variables that I do want to leak through, especially many coming from the batch system.

    • It's even possible that the default for new images can change.

  • Old images retain existing behavior.
  • Allow the sysadmin to control the default behavior via config files.
bbockelm picture bbockelm on 5 Jan 2017

Well, I can easily answer both requests with usage tips and tell me if anyone sees a better way to implement:

This will change the contents/location of the home directory to whatever you have inside of the directory ~/virtualized_home.

$ singularity shell -H ~/virtualized_home container.img

This will run Singularity in a clean environment with no unnecessary variables:

$ env -i singularity shell container.img

And if you combine the two, I think you will have exactly what you need. But... Let me know if you think there is a better way to implement either of these or if I misunderstood and it doesn't suitably handle your needs.

gmkurtzer picture gmkurtzer on 5 Jan 2017

@bbockelm I liek the idea of specifying how home mount and environemnt variables are handled to be done in the container image. Having said that for images imported from Docker Hub I would make ignoring host ENV and HOME a default that can be changed on the command line.

@gmkurtzer the -H flag does not seem to work for me:

[chrisgor@sherlock-ln02 login_node ~]$ singularity --version
2.2
[chrisgor@sherlock-ln02 login_node ~]$ ls /scratch/users/chrisgor/
beast_work  bla  masked.nii.gz  neurovault_atlasing  rewardBeastBIDS_nofmap  work
[chrisgor@sherlock-ln02 login_node ~]$ singularity shell -H /scratch/users/chrisgor/ /share/PI/russpold/singularity_images/poldracklab_fmriprep_v6-2016-12-09-98122cb3e276.img
Singularity: Invoking an interactive shell within container...

bash: module: command not found
chrisgor@sherlock-ln02:~$ ls $HOME
12_smooth_res4d_std_noMC_2mmMNI152_GM.nii.gz  Linux_X_64.tar.bz2                               fmri_fsl                   hcp.job.err        java.log.19252         matlab               rename.sh                  software
FreeSurferPipeline.sh.e14190                  Miniconda-latest-Linux-x86_64.sh                 fmriprep                   hcp.job.out        java.log.30341         ndmg1.batch          rest_mcf_trans.nii.gz      spm12-master
FreeSurferPipeline.sh.e14230                  Searchlight_vs_chance_Figure_4_spmT_0001.nii.gz  freesurfer1.job.err        hcp_all.batch      java.log.36616         ndmg1.job.err        rh.pial.deformed.out       sub-01_ses-1_task-rest_acq-fullbrain_run-1_physio.tsv
FreeSurferPipeline.sh.e45749                  agave_test.err                                   freesurfer1.job.out        hcp_all.job.err    java.log.368           ndmg1.job.out        rh.white.deformed.out      sub-01_ses-1_task-rest_acq-fullbrain_run-1_physio.tsv.gz
FreeSurferPipeline.sh.o14190                  agave_test.out                                   freesurfer1_trace.job.err  hcp_all.job.out    java.log.41894         ndmg2.batch          run.py                     sub-387.job.err
FreeSurferPipeline.sh.o14230                  agave_test.sbatch                                freesurfer1_trace.job.out  hcp_fmriV.batch    java.log.43753         ndmg2.job.err        run.py.1                   sub-387.job.out
FreeSurferPipeline.sh.o45749                  atlas_analysis                                   freesurfer2.job.err        hcp_fmris.batch    job.sh                 ndmg2.job.out        sbatch_examples            talairach_with_skull.log
ICBM2009_fs.batch                             atlas_analysis.batch                             freesurfer2.job.out        hcp_fmris.job.err  job_freesurfer.sh      nipype               singularity-thin-demo.img  tsvgzread.m
ICBM2009_fs6.batch                            atlas_fake_data.job.err                          freesurfer_group.job.err   hcp_fmris.job.out  job_mri_nu_correct.sh  niworkflows          singularity_write          workdir
ICBM2009_fs_withoutT2.batch                   atlas_fake_data.job.out                          freesurfer_group.job.out   hcp_fmriv.job.err  jobs                   out.nii              slurm.err
ICBM2009c_fs.job.err                          atlases                                          fs1.batch                  hcp_fmriv.job.out  lh.pial.deformed.out   preprocess_beast     slurm.out
ICBM2009c_fs.job.out                          beast_sub-387.sbatch                             fs1_trace.batch            hcp_post.batch     lh.white.deformed.out  preprocess_ds000005  slurm_freesurfer.err
ICBM2009c_fs6.job.err                         c3d-nightly-Linux-x86_64.tar.gz                  fs2.batch                  hcp_post.job.err   log_error.log          preprocess_ds000102  slurm_freesurfer.out
ICBM2009c_fs6.job.out                         data                                             fs_group.batch             hcp_post.job.out   log_noerror.log        preprocess_ds000107  slurm_test.batch
ICBM2009c_fs_noT2.job.err                     docker2aci                                       gradunwarp                 java.log.10441     macjob.py              preprocess_ds000109  slurm_test.job.err
ICBM2009c_fs_noT2.job.out                     downloads                                        hcp.batch                  java.log.19063     master.zip             read_tsv_gz.m        slurm_test.job.out
chrisgorgo picture chrisgorgo on 5 Jan 2017

What is in /scratch/users/chrisgor/ and how does that compare to your $HOME?

hrmm... It does seem that the your dot files are still being found, interesting. Can you send the --debug output of that?

gmkurtzer picture gmkurtzer on 5 Jan 2017

Could it have something to do with the fact that sherlock automatically mounts scratch directories?

# BIND PATH: [STRING]
# DEFAULT: Undefined
# Define a list of files/directories that should be made available from within
# the container. The file or directory must exist within the container on
# which to attach to. you can specify a different source and destination
# path (respectively) with a colon; otherwise source and dest are the same.
#bind path = /etc/singularity/default-nsswitch.conf:/etc/nsswitch.conf
#bind path = /opt
#bind path = /scratch
bind path = /etc/hosts
bind path = /dev
bind path = /tmp
bind path = /var/tmp
bind path = /home
bind path = /share/PI
bind path = /scratch
bind path = /local-scratch

also these might be relevant:

# MOUNT HOME: [BOOL]
# DEFAULT: yes
# Should we automatically determine the calling user's home directory and
# attempt to mount it's base path into the container? If the --contain option
# is used, the home directory will be created within the session directory or
# can be overridden with the SINGULARITY_HOME or SINGULARITY_WORKDIR
# environment variables (or their corresponding command line options).
mount home = yes



# CONFIG RESOLV_CONF: [BOOL]
# DEFAULT: yes
# If there is a bind point within the container, use the host's
# /etc/resolv.conf.
config resolv_conf = yes
vsoch picture vsoch on 5 Jan 2017

Yes, there could be something weird going on there actually... @chrisfilo, can you create a new empty directory in /scratch/users/chrisgor/ and use that for the -H option?

gmkurtzer picture gmkurtzer on 5 Jan 2017 
[chrisgor@sherlock-ln02 login_node ~]$ ls $HOME
12_smooth_res4d_std_noMC_2mmMNI152_GM.nii.gz  freesurfer1_trace.job.err     hcp_all.job.err        ICBM2009c_fs_noT2.job.err    lh.white.deformed.out             preprocess_ds000005                              slurm.out
agave_test.err                                freesurfer1_trace.job.out     hcp_all.job.out        ICBM2009c_fs_noT2.job.out    Linux_X_64.tar.bz2                preprocess_ds000102                              slurm_test.batch
agave_test.out                                freesurfer2.job.err           hcp.batch              ICBM2009_fs6.batch           log_error.log                     preprocess_ds000107                              slurm_test.job.err
agave_test.sbatch                             freesurfer2.job.out           hcp_fmris.batch        ICBM2009_fs.batch            log_noerror.log                   preprocess_ds000109                              slurm_test.job.out
atlas_analysis                                freesurfer_group.job.err      hcp_fmris.job.err      ICBM2009_fs_withoutT2.batch  macjob.py                         read_tsv_gz.m                                    software
atlas_analysis.batch                          freesurfer_group.job.out      hcp_fmris.job.out      java.log.10441               master.zip                        rename.sh                                        spm12-master
atlases                                       FreeSurferPipeline.sh.e14190  hcp_fmriV.batch        java.log.19063               matlab                            rest_mcf_trans.nii.gz                            sub-01_ses-1_task-rest_acq-fullbrain_run-1_physio.tsv
atlas_fake_data.job.err                       FreeSurferPipeline.sh.e14230  hcp_fmriv.job.err      java.log.19252               Miniconda-latest-Linux-x86_64.sh  rh.pial.deformed.out                             sub-01_ses-1_task-rest_acq-fullbrain_run-1_physio.tsv.gz
atlas_fake_data.job.out                       FreeSurferPipeline.sh.e45749  hcp_fmriv.job.out      java.log.30341               ndmg1.batch                       rh.white.deformed.out                            sub-387.job.err
beast_sub-387.sbatch                          FreeSurferPipeline.sh.o14190  hcp.job.err            java.log.36616               ndmg1.job.err                     run.py                                           sub-387.job.out
c3d-nightly-Linux-x86_64.tar.gz               FreeSurferPipeline.sh.o14230  hcp.job.out            java.log.368                 ndmg1.job.out                     run.py.1                                         talairach_with_skull.log
data                                          FreeSurferPipeline.sh.o45749  hcp_post.batch         java.log.41894               ndmg2.batch                       sbatch_examples                                  tsvgzread.m
docker2aci                                    fs1.batch                     hcp_post.job.err       java.log.43753               ndmg2.job.err                     Searchlight_vs_chance_Figure_4_spmT_0001.nii.gz  workdir
downloads                                     fs1_trace.batch               hcp_post.job.out       job_freesurfer.sh            ndmg2.job.out                     singularity-thin-demo.img
fmri_fsl                                      fs2.batch                     ICBM2009c_fs6.job.err  job_mri_nu_correct.sh        nipype                            singularity_write
fmriprep                                      fs_group.batch                ICBM2009c_fs6.job.out  jobs                         niworkflows                       slurm.err
freesurfer1.job.err                           gradunwarp                    ICBM2009c_fs.job.err   job.sh                       out.nii                           slurm_freesurfer.err
freesurfer1.job.out                           hcp_all.batch                 ICBM2009c_fs.job.out   lh.pial.deformed.out         preprocess_beast                  slurm_freesurfer.out
[chrisgor@sherlock-ln02 login_node ~]$ ls /scratch/users/chrisgor/
beast_work  bla  masked.nii.gz  neurovault_atlasing  rewardBeastBIDS_nofmap  work

[chrisgor@sherlock-ln02 login_node ~]$ singularity --debug shell -H /scratch/users/chrisgor/ /share/PI/russpold/singularity_images/poldracklab_fmriprep_v6-2016-12-09-98122cb3e276.img
enabling debugging
ending argument loop
Exec'ing: /share/sw/free/singularity/2.2/libexec/singularity/cli/shell.exec -HVERBOSE [U=265085,P=24642] message.c:52:init()                        : Set messagelevel to: 5
DEBUG   [U=265085,P=24642] privilege.c:66:singularity_priv_init()     : Called singularity_priv_init(void)
DEBUG   [U=265085,P=24642] privilege.c:131:singularity_priv_init()    : Returning singularity_priv_init(void)
DEBUG   [U=265085,P=24642] privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24642] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
VERBOSE [U=265085,P=24642] sexec.c:73:main()                          : Running NON-SUID program workflow
DEBUG   [U=265085,P=24642] sexec.c:75:main()                          : Checking program has appropriate permissions
VERBOSE [U=265085,P=24642] config_parser.c:43:singularity_config_open(): Opening configuration file: /share/sw/free/singularity/2.2/etc/singularity/singularity.conf
DEBUG   [U=265085,P=24642] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
VERBOSE [U=265085,P=24642] sexec.c:85:main()                          : Checking that we are allowed to run as SUID
DEBUG   [U=265085,P=24642] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(allow setuid, 1)
DEBUG   [U=265085,P=24642] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(allow setuid)
VERBOSE [U=265085,P=24642] config_parser.c:91:singularity_config_get_value(): Got config key allow setuid (= 'yes')
DEBUG   [U=265085,P=24642] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(allow setuid, 1) = 1
VERBOSE [U=265085,P=24642] sexec.c:87:main()                          : Checking if we were requested to run as NOSUID by user
DEBUG   [U=265085,P=24642] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_NOSUID
VERBOSE [U=265085,P=24642] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_NOSUID
VERBOSE [U=265085,P=24642] sexec.c:92:main()                          : Invoking SUID sexec: /share/sw/free/singularity/2.2/libexec/singularity/sexec-suid
VERBOSE [U=0,P=24642]      message.c:52:init()                        : Set messagelevel to: 5
DEBUG   [U=0,P=24642]      privilege.c:66:singularity_priv_init()     : Called singularity_priv_init(void)
DEBUG   [U=0,P=24642]      privilege.c:131:singularity_priv_init()    : Returning singularity_priv_init(void)
DEBUG   [U=0,P=24642]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24642] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
VERBOSE [U=265085,P=24642] sexec.c:46:main()                          : Running SUID program workflow
VERBOSE [U=265085,P=24642] sexec.c:48:main()                          : Checking program has appropriate permissions
VERBOSE [U=265085,P=24642] sexec.c:53:main()                          : Checking configuration file is properly owned by root
VERBOSE [U=265085,P=24642] config_parser.c:43:singularity_config_open(): Opening configuration file: /share/sw/free/singularity/2.2/etc/singularity/singularity.conf
DEBUG   [U=265085,P=24642] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
VERBOSE [U=265085,P=24642] sexec.c:62:main()                          : Checking that we are allowed to run as SUID
DEBUG   [U=265085,P=24642] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(allow setuid, 1)
DEBUG   [U=265085,P=24642] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(allow setuid)
VERBOSE [U=265085,P=24642] config_parser.c:91:singularity_config_get_value(): Got config key allow setuid (= 'yes')
DEBUG   [U=265085,P=24642] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(allow setuid, 1) = 1
VERBOSE [U=265085,P=24642] sexec.c:67:main()                          : Checking if we were requested to run as NOSUID by user
DEBUG   [U=265085,P=24642] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_NOSUID
VERBOSE [U=265085,P=24642] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_NOSUID
DEBUG   [U=265085,P=24642] util/util.c:102:envar_path()               : Checking environment variable is valid path: 'SINGULARITY_IMAGE'
VERBOSE [U=265085,P=24642] util/util.c:50:envar()                     : Checking input from environment: 'SINGULARITY_IMAGE'
DEBUG   [U=265085,P=24642] util/util.c:52:envar()                     : Checking environment variable is defined: SINGULARITY_IMAGE
DEBUG   [U=265085,P=24642] util/util.c:58:envar()                     : Checking environment variable length (<= 4096): SINGULARITY_IMAGE
DEBUG   [U=265085,P=24642] util/util.c:64:envar()                     : Checking environment variable has allowed characters: SINGULARITY_IMAGE
VERBOSE [U=265085,P=24642] util/util.c:87:envar()                     : Obtained input from environment 'SINGULARITY_IMAGE' = '/share/PI/russpold/singularity_images/poldracklab_fmriprep_v6-2016-12-09-98122cb3e276.img'
VERBOSE [U=265085,P=24642] util/util.c:50:envar()                     : Checking input from environment: 'SINGULARITY_COMMAND'
DEBUG   [U=265085,P=24642] util/util.c:52:envar()                     : Checking environment variable is defined: SINGULARITY_COMMAND
DEBUG   [U=265085,P=24642] util/util.c:58:envar()                     : Checking environment variable length (<= 10): SINGULARITY_COMMAND
DEBUG   [U=265085,P=24642] util/util.c:64:envar()                     : Checking environment variable has allowed characters: SINGULARITY_COMMAND
VERBOSE [U=265085,P=24642] util/util.c:87:envar()                     : Obtained input from environment 'SINGULARITY_COMMAND' = 'shell'
DEBUG   [U=265085,P=24642] action.c:55:singularity_action_init()      : Checking on action to run
DEBUG   [U=265085,P=24642] action.c:63:singularity_action_init()      : Setting action to: shell
DEBUG   [U=265085,P=24642] action.c:95:singularity_action_init()      : Getting current working directory path string
DEBUG   [U=265085,P=24642] rootfs.c:71:singularity_rootfs_init()      : Checking on container source type
DEBUG   [U=265085,P=24642] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24642] rootfs.c:80:singularity_rootfs_init()      : Figuring out where to mount Singularity container
DEBUG   [U=265085,P=24642] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(container dir)
VERBOSE [U=265085,P=24642] config_parser.c:91:singularity_config_get_value(): Got config key container dir (= '/var/singularity/mnt')
DEBUG   [U=265085,P=24642] rootfs.c:86:singularity_rootfs_init()      : Set image mount path to: /var/singularity/mnt
DEBUG   [U=265085,P=24642] image.c:52:rootfs_image_init()             : Inializing container rootfs image subsystem
DEBUG   [U=265085,P=24642] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_WRITABLE
VERBOSE [U=265085,P=24642] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_WRITABLE
VERBOSE [U=265085,P=24642] image-util.c:48:singularity_image_check()  : Checking file is a Singularity image
DEBUG   [U=265085,P=24642] image-util.c:59:singularity_image_check()  : Checking if first line matches key
VERBOSE [U=265085,P=24642] image-util.c:62:singularity_image_check()  : File is a valid Singularity image
DEBUG   [U=265085,P=24642] sessiondir.c:60:singularity_sessiondir_init(): Checking Singularity configuration for 'sessiondir prefix'
DEBUG   [U=265085,P=24642] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24642] util/util.c:102:envar_path()               : Checking environment variable is valid path: 'SINGULARITY_SESSIONDIR'
VERBOSE [U=265085,P=24642] util/util.c:50:envar()                     : Checking input from environment: 'SINGULARITY_SESSIONDIR'
DEBUG   [U=265085,P=24642] util/util.c:52:envar()                     : Checking environment variable is defined: SINGULARITY_SESSIONDIR
VERBOSE [U=265085,P=24642] util/util.c:54:envar()                     : Environment variable is NULL: SINGULARITY_SESSIONDIR
DEBUG   [U=265085,P=24642] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(sessiondir prefix)
DEBUG   [U=265085,P=24642] config_parser.c:99:singularity_config_get_value(): No configuration file entry found for 'sessiondir prefix'
DEBUG   [U=265085,P=24642] sessiondir.c:75:singularity_sessiondir_init(): Set sessiondir to: /tmp/.singularity-session-265085.36.5089639556
DEBUG   [U=265085,P=24642] util/file.c:245:s_mkpath()                 : Creating directory: /tmp/.singularity-session-265085.36.5089639556
DEBUG   [U=265085,P=24642] sessiondir.c:91:singularity_sessiondir_init(): Opening sessiondir file descriptor
DEBUG   [U=265085,P=24642] sessiondir.c:97:singularity_sessiondir_init(): Setting shared flock() on session directory
DEBUG   [U=265085,P=24642] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_NOSESSIONCLEANUP
VERBOSE [U=265085,P=24642] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_NOSESSIONCLEANUP
DEBUG   [U=265085,P=24642] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_NOCLEANUP
VERBOSE [U=265085,P=24642] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_NOCLEANUP
VERBOSE [U=265085,P=24642] fork.c:74:singularity_fork()               : Forking child process
VERBOSE [U=265085,P=24642] fork.c:90:singularity_fork()               : Hello from parent process
DEBUG   [U=265085,P=24642] fork.c:109:singularity_fork()              : Assigning sigaction()s
DEBUG   [U=265085,P=24642] fork.c:140:singularity_fork()              : Creating generic signal pipes
DEBUG   [U=265085,P=24642] fork.c:148:singularity_fork()              : Creating sigcld signal pipes
DEBUG   [U=265085,P=24642] fork.c:170:singularity_fork()              : Waiting on signal from watchdog
VERBOSE [U=265085,P=24651] fork.c:78:singularity_fork()               : Hello from child process
DEBUG   [U=265085,P=24651] fork.c:81:singularity_fork()               : Closing watchdog write pipe
DEBUG   [U=265085,P=24651] fork.c:86:singularity_fork()               : Child process is returning control to process thread
DEBUG   [U=265085,P=24651] ns.c:45:singularity_ns_unshare()           : Unsharing all namespaces
VERBOSE [U=265085,P=24651] user.c:61:singularity_ns_user_unshare()    : Not virtualizing user namespace: running SUID root
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(allow pid ns, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(allow pid ns)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key allow pid ns (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(allow pid ns, 1) = 1
DEBUG   [U=265085,P=24651] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_UNSHARE_PID
VERBOSE [U=265085,P=24651] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_UNSHARE_PID
VERBOSE [U=265085,P=24651] pid.c:59:singularity_ns_pid_unshare()      : Not virtualizing PID namespace on user request
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(mount slave, 0)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(mount slave)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key mount slave (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(mount slave, 0) = 1
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
DEBUG   [U=0,P=24651]      mnt.c:54:singularity_ns_mnt_unshare()      : Virtualizing FS namespace
DEBUG   [U=0,P=24651]      mnt.c:61:singularity_ns_mnt_unshare()      : Virtualizing mount namespace
DEBUG   [U=0,P=24651]      mnt.c:70:singularity_ns_mnt_unshare()      : Making mounts slave
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(allow ipc ns, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(allow ipc ns)
DEBUG   [U=265085,P=24651] config_parser.c:99:singularity_config_get_value(): No configuration file entry found for 'allow ipc ns'
DEBUG   [U=265085,P=24651] config_parser.c:126:singularity_config_get_bool(): Undefined configuration for 'allow ipc ns', returning default: yes
DEBUG   [U=265085,P=24651] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_UNSHARE_IPC
VERBOSE [U=265085,P=24651] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_UNSHARE_IPC
VERBOSE [U=265085,P=24651] ipc.c:59:singularity_ns_ipc_unshare()      : Not virtualizing IPC namespace on user request
DEBUG   [U=265085,P=24651] rootfs.c:117:singularity_rootfs_mount()    : Mounting image
DEBUG   [U=265085,P=24651] rootfs.c:119:singularity_rootfs_mount()    : Checking for rootfs_source directory: /var/singularity/mnt/source
DEBUG   [U=265085,P=24651] rootfs.c:130:singularity_rootfs_mount()    : Checking for overlay_mount directory: /var/singularity/mnt/overlay
DEBUG   [U=265085,P=24651] rootfs.c:141:singularity_rootfs_mount()    : Checking for overlay_final directory: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] image.c:119:rootfs_image_mount()           : Binding image to loop device
DEBUG   [U=265085,P=24651] sessiondir.c:149:singularity_sessiondir_get(): Returning: /tmp/.singularity-session-265085.36.5089639556
DEBUG   [U=265085,P=24651] loop-control.c:65:singularity_loop_bind()  : Opening image loop device file: /tmp/.singularity-session-265085.36.5089639556/image_loop_dev
DEBUG   [U=265085,P=24651] loop-control.c:71:singularity_loop_bind()  : Requesting exclusive flock() on loop_dev lockfile
DEBUG   [U=265085,P=24651] loop-control.c:94:singularity_loop_bind()  : Calculating image offset
VERBOSE [U=265085,P=24651] image-util.c:77:singularity_image_offset() : Calculating image offset
VERBOSE [U=265085,P=24651] image-util.c:86:singularity_image_offset() : Found image at an offset of 31 bytes
DEBUG   [U=265085,P=24651] image-util.c:91:singularity_image_offset() : Returning image_offset(image_fp) = 31
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
DEBUG   [U=0,P=24651]      loop-control.c:101:singularity_loop_bind() : Finding next available loop device...
VERBOSE [U=0,P=24651]      loop-control.c:133:singularity_loop_bind() : Found avaialble loop device: /dev/loop0
DEBUG   [U=0,P=24651]      loop-control.c:135:singularity_loop_bind() : Setting loop device flags
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
VERBOSE [U=265085,P=24651] loop-control.c:145:singularity_loop_bind() : Using loop device: /dev/loop0
DEBUG   [U=265085,P=24651] loop-control.c:147:singularity_loop_bind() : Writing active loop device name (/dev/loop0) to loop file cache: /tmp/.singularity-session-265085.36.5089639556/image_loop_dev
DEBUG   [U=265085,P=24651] util/file.c:327:fileput()                  : Called fileput(/tmp/.singularity-session-265085.36.5089639556/image_loop_dev, /dev/loop0)
DEBUG   [U=265085,P=24651] loop-control.c:153:singularity_loop_bind() : Resetting exclusive flock() to shared on image_loop_file
DEBUG   [U=265085,P=24651] loop-control.c:156:singularity_loop_bind() : Returning singularity_loop_bind(image_fp) = loop_fp
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      image.c:136:rootfs_image_mount()           : Mounting image in read/only
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      rootfs.c:222:singularity_rootfs_mount()    : Binding the ROOTFS_SOURCE to OVERLAY_FINAL (/var/singularity/mnt/source->/var/singularity/mnt/final)
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] rootfs.c:235:singularity_rootfs_check()    : Checking if container has /bin/sh...
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] sessiondir.c:149:singularity_sessiondir_get(): Returning: /tmp/.singularity-session-265085.36.5089639556
DEBUG   [U=265085,P=24651] passwd.c:53:singularity_file_passwd()      : Called singularity_file_passwd_create()
DEBUG   [U=265085,P=24651] passwd.c:70:singularity_file_passwd()      : Checking configuration option: 'config passwd'
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(config passwd, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(config passwd)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key config passwd (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(config passwd, 1) = 1
VERBOSE [U=265085,P=24651] passwd.c:80:singularity_file_passwd()      : Checking for template passwd file: /var/singularity/mnt/final/etc/passwd
VERBOSE [U=265085,P=24651] passwd.c:86:singularity_file_passwd()      : Creating template of /etc/passwd
DEBUG   [U=265085,P=24651] util/file.c:277:copy_file()                : Called copy_file(/var/singularity/mnt/final/etc/passwd, /tmp/.singularity-session-265085.36.5089639556/passwd)
DEBUG   [U=265085,P=24651] util/file.c:284:copy_file()                : Opening source file: /var/singularity/mnt/final/etc/passwd
DEBUG   [U=265085,P=24651] util/file.c:290:copy_file()                : Opening destination file: /tmp/.singularity-session-265085.36.5089639556/passwd
DEBUG   [U=265085,P=24651] util/file.c:297:copy_file()                : Calling fstat() on source file descriptor: 11
DEBUG   [U=265085,P=24651] util/file.c:303:copy_file()                : Cloning permission string of source to dest
DEBUG   [U=265085,P=24651] util/file.c:309:copy_file()                : Copying file data...
DEBUG   [U=265085,P=24651] util/file.c:314:copy_file()                : Done copying data, closing file pointers
DEBUG   [U=265085,P=24651] util/file.c:318:copy_file()                : Returning copy_file(/var/singularity/mnt/final/etc/passwd, /tmp/.singularity-session-265085.36.5089639556/passwd) = 0
DEBUG   [U=265085,P=24651] passwd.c:92:singularity_file_passwd()      : Opening the template passwd file: /tmp/.singularity-session-265085.36.5089639556/passwd
VERBOSE [U=265085,P=24651] passwd.c:98:singularity_file_passwd()      : Creating template passwd file and appending user data
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] file-bind.c:41:container_file_bind()       : Called file_bind(/tmp/.singularity-session-265085.36.5089639556/passwd, /etc/passwd()
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      file-bind.c:61:container_file_bind()       : Binding file '/tmp/.singularity-session-265085.36.5089639556/passwd' to '/var/singularity/mnt/final/etc/passwd'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] sessiondir.c:149:singularity_sessiondir_get(): Returning: /tmp/.singularity-session-265085.36.5089639556
DEBUG   [U=265085,P=24651] group.c:58:singularity_file_group()        : Called singularity_file_group_create()
DEBUG   [U=265085,P=24651] group.c:75:singularity_file_group()        : Checking configuration option: 'config group'
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(config group, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(config group)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key config group (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(config group, 1) = 1
VERBOSE [U=265085,P=24651] group.c:102:singularity_file_group()       : Creating template of /etc/group for containment
DEBUG   [U=265085,P=24651] util/file.c:277:copy_file()                : Called copy_file(/var/singularity/mnt/final/etc/group, /tmp/.singularity-session-265085.36.5089639556/group)
DEBUG   [U=265085,P=24651] util/file.c:284:copy_file()                : Opening source file: /var/singularity/mnt/final/etc/group
DEBUG   [U=265085,P=24651] util/file.c:290:copy_file()                : Opening destination file: /tmp/.singularity-session-265085.36.5089639556/group
DEBUG   [U=265085,P=24651] util/file.c:297:copy_file()                : Calling fstat() on source file descriptor: 12
DEBUG   [U=265085,P=24651] util/file.c:303:copy_file()                : Cloning permission string of source to dest
DEBUG   [U=265085,P=24651] util/file.c:309:copy_file()                : Copying file data...
DEBUG   [U=265085,P=24651] util/file.c:314:copy_file()                : Done copying data, closing file pointers
DEBUG   [U=265085,P=24651] util/file.c:318:copy_file()                : Returning copy_file(/var/singularity/mnt/final/etc/group, /tmp/.singularity-session-265085.36.5089639556/group) = 0
VERBOSE [U=265085,P=24651] group.c:115:singularity_file_group()       : Updating group file with user info
DEBUG   [U=265085,P=24651] group.c:127:singularity_file_group()       : Getting supplementary group info
VERBOSE [U=265085,P=24651] group.c:139:singularity_file_group()       : Found supplementary group membership in: 10000
VERBOSE [U=265085,P=24651] group.c:140:singularity_file_group()       : Adding user's supplementary group ('sherlock_users') info to template group file
DEBUG   [U=265085,P=24651] group.c:131:singularity_file_group()       : Skipping duplicate supplementary group
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] file-bind.c:41:container_file_bind()       : Called file_bind(/tmp/.singularity-session-265085.36.5089639556/group, /etc/group()
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      file-bind.c:61:container_file_bind()       : Binding file '/tmp/.singularity-session-265085.36.5089639556/group' to '/var/singularity/mnt/final/etc/group'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] resolvconf.c:47:singularity_file_resolvconf(): Checking configuration option
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(config resolv_conf, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(config resolv_conf)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key config resolv_conf (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(config resolv_conf, 1) = 1
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] file-bind.c:41:container_file_bind()       : Called file_bind(/etc/resolv.conf, /etc/resolv.conf()
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      file-bind.c:61:container_file_bind()       : Binding file '/etc/resolv.conf' to '/var/singularity/mnt/final/etc/resolv.conf'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(mount hostfs, 0)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(mount hostfs)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key mount hostfs (= 'no')
DEBUG   [U=265085,P=24651] config_parser.c:118:singularity_config_get_bool(): Return singularity_config_get_bool(mount hostfs, 0) = 0
DEBUG   [U=265085,P=24651] hostfs.c:53:singularity_mount_hostfs()     : Not mounting host file systems per configuration
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_CONTAIN
VERBOSE [U=265085,P=24651] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_CONTAIN
DEBUG   [U=265085,P=24651] binds.c:48:singularity_mount_binds()       : Checking configuration file for 'bind path'
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key bind path (= '/etc/hosts')
VERBOSE [U=265085,P=24651] binds.c:63:singularity_mount_binds()       : Found 'bind path' = /etc/hosts, /etc/hosts
DEBUG   [U=265085,P=24651] binds.c:70:singularity_mount_binds()       : Checking if bind point is already mounted: /etc/hosts
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      binds.c:112:singularity_mount_binds()      : Binding '/etc/hosts' to '/var/singularity/mnt/final//etc/hosts'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key bind path (= '/dev')
VERBOSE [U=265085,P=24651] binds.c:63:singularity_mount_binds()       : Found 'bind path' = /dev, /dev
DEBUG   [U=265085,P=24651] binds.c:70:singularity_mount_binds()       : Checking if bind point is already mounted: /dev
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      binds.c:112:singularity_mount_binds()      : Binding '/dev' to '/var/singularity/mnt/final//dev'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key bind path (= '/tmp')
VERBOSE [U=265085,P=24651] binds.c:63:singularity_mount_binds()       : Found 'bind path' = /tmp, /tmp
DEBUG   [U=265085,P=24651] binds.c:70:singularity_mount_binds()       : Checking if bind point is already mounted: /tmp
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      binds.c:112:singularity_mount_binds()      : Binding '/tmp' to '/var/singularity/mnt/final//tmp'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key bind path (= '/var/tmp')
VERBOSE [U=265085,P=24651] binds.c:63:singularity_mount_binds()       : Found 'bind path' = /var/tmp, /var/tmp
DEBUG   [U=265085,P=24651] binds.c:70:singularity_mount_binds()       : Checking if bind point is already mounted: /var/tmp
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      binds.c:112:singularity_mount_binds()      : Binding '/var/tmp' to '/var/singularity/mnt/final//var/tmp'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key bind path (= '/home')
VERBOSE [U=265085,P=24651] binds.c:63:singularity_mount_binds()       : Found 'bind path' = /home, /home
DEBUG   [U=265085,P=24651] binds.c:70:singularity_mount_binds()       : Checking if bind point is already mounted: /home
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      binds.c:112:singularity_mount_binds()      : Binding '/home' to '/var/singularity/mnt/final//home'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key bind path (= '/share/PI')
VERBOSE [U=265085,P=24651] binds.c:63:singularity_mount_binds()       : Found 'bind path' = /share/PI, /share/PI
DEBUG   [U=265085,P=24651] binds.c:70:singularity_mount_binds()       : Checking if bind point is already mounted: /share/PI
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      binds.c:112:singularity_mount_binds()      : Binding '/share/PI' to '/var/singularity/mnt/final//share/PI'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key bind path (= '/scratch')
VERBOSE [U=265085,P=24651] binds.c:63:singularity_mount_binds()       : Found 'bind path' = /scratch, /scratch
DEBUG   [U=265085,P=24651] binds.c:70:singularity_mount_binds()       : Checking if bind point is already mounted: /scratch
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      binds.c:112:singularity_mount_binds()      : Binding '/scratch' to '/var/singularity/mnt/final//scratch'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key bind path (= '/local-scratch')
VERBOSE [U=265085,P=24651] binds.c:63:singularity_mount_binds()       : Found 'bind path' = /local-scratch, /local-scratch
DEBUG   [U=265085,P=24651] binds.c:70:singularity_mount_binds()       : Checking if bind point is already mounted: /local-scratch
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      binds.c:112:singularity_mount_binds()      : Binding '/local-scratch' to '/var/singularity/mnt/final//local-scratch'
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(bind path)
DEBUG   [U=265085,P=24651] config_parser.c:99:singularity_config_get_value(): No configuration file entry found for 'bind path'
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] kernelfs.c:43:singularity_mount_kernelfs() : Checking configuration file for 'mount proc'
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(mount proc, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(mount proc)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key mount proc (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(mount proc, 1) = 1
DEBUG   [U=265085,P=24651] pid.c:46:singularity_ns_pid_enabled()      : Checking PID namespace enabled: -1
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      kernelfs.c:57:singularity_mount_kernelfs() : Bind mounting /proc
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] kernelfs.c:73:singularity_mount_kernelfs() : Checking configuration file for 'mount sys'
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(mount sys, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(mount sys)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key mount sys (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(mount sys, 1) = 1
DEBUG   [U=265085,P=24651] user.c:45:singularity_ns_user_enabled()    : Checking user namespace enabled: -1
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      kernelfs.c:79:singularity_mount_kernelfs() : Mounting /sys
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(mount dev)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key mount dev (= 'yes')
DEBUG   [U=265085,P=24651] dev.c:91:singularity_mount_dev()           : Checking configuration file for 'mount dev'
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(mount dev, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(mount dev)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key mount dev (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(mount dev, 1) = 1
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      dev.c:96:singularity_mount_dev()           : Bind mounting /dev
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(mount tmp, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(mount tmp)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key mount tmp (= 'no')
DEBUG   [U=265085,P=24651] config_parser.c:118:singularity_config_get_bool(): Return singularity_config_get_bool(mount tmp, 1) = 0
VERBOSE [U=265085,P=24651] tmp.c:48:singularity_mount_tmp()           : Skipping tmp dir mounting (per config)
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] sessiondir.c:149:singularity_sessiondir_get(): Returning: /tmp/.singularity-session-265085.36.5089639556
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(mount home, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(mount home)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key mount home (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(mount home, 1) = 1
DEBUG   [U=265085,P=24651] home.c:69:singularity_mount_home()         : Obtaining user's homedir
DEBUG   [U=265085,P=24651] home.c:72:singularity_mount_home()         : Checking if home directory is already mounted: /home/chrisgor
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:65:check_mounted()            : Mountpoint is already mounted: /home/chrisgor
VERBOSE [U=265085,P=24651] home.c:74:singularity_mount_home()         : Not mounting home directory (already mounted in container): /home/chrisgor
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] userbinds.c:45:singularity_mount_userbinds(): Checking for environment variable 'SINGULARITY_BINDPATH'
DEBUG   [U=265085,P=24651] util/util.c:102:envar_path()               : Checking environment variable is valid path: 'SINGULARITY_BINDPATH'
VERBOSE [U=265085,P=24651] util/util.c:50:envar()                     : Checking input from environment: 'SINGULARITY_BINDPATH'
DEBUG   [U=265085,P=24651] util/util.c:52:envar()                     : Checking environment variable is defined: SINGULARITY_BINDPATH
VERBOSE [U=265085,P=24651] util/util.c:54:envar()                     : Environment variable is NULL: SINGULARITY_BINDPATH
DEBUG   [U=265085,P=24651] userbinds.c:148:singularity_mount_userbinds(): No user bind mounts specified.
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] scratch.c:53:singularity_mount_scratch()   : Getting SINGULARITY_SCRATCHDIR from environment
DEBUG   [U=265085,P=24651] util/util.c:102:envar_path()               : Checking environment variable is valid path: 'SINGULARITY_SCRATCHDIR'
VERBOSE [U=265085,P=24651] util/util.c:50:envar()                     : Checking input from environment: 'SINGULARITY_SCRATCHDIR'
DEBUG   [U=265085,P=24651] util/util.c:52:envar()                     : Checking environment variable is defined: SINGULARITY_SCRATCHDIR
VERBOSE [U=265085,P=24651] util/util.c:54:envar()                     : Environment variable is NULL: SINGULARITY_SCRATCHDIR
DEBUG   [U=265085,P=24651] scratch.c:55:singularity_mount_scratch()   : Not mounting scratch directory: Not requested
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] cwd.c:47:singularity_mount_cwd()           : Checking to see if we should mount current working directory
DEBUG   [U=265085,P=24651] cwd.c:49:singularity_mount_cwd()           : Getting current working directory
DEBUG   [U=265085,P=24651] cwd.c:55:singularity_mount_cwd()           : Checking configuration file for 'user bind control'
DEBUG   [U=265085,P=24651] config_parser.c:62:singularity_config_rewind(): Rewinding configuration file
DEBUG   [U=265085,P=24651] config_parser.c:107:singularity_config_get_bool(): Called singularity_config_get_bool(user bind control, 1)
DEBUG   [U=265085,P=24651] config_parser.c:80:singularity_config_get_value(): Called singularity_config_get_value(user bind control)
VERBOSE [U=265085,P=24651] config_parser.c:91:singularity_config_get_value(): Got config key user bind control (= 'yes')
DEBUG   [U=265085,P=24651] config_parser.c:113:singularity_config_get_bool(): Return singularity_config_get_bool(user bind control, 1) = 1
DEBUG   [U=265085,P=24651] cwd.c:67:singularity_mount_cwd()           : Checking for contain option
DEBUG   [U=265085,P=24651] util/util.c:92:envar_defined()             : Checking if environment variable is defined: SINGULARITY_CONTAIN
VERBOSE [U=265085,P=24651] util/util.c:94:envar_defined()             : Environment variable is undefined: SINGULARITY_CONTAIN
DEBUG   [U=265085,P=24651] cwd.c:73:singularity_mount_cwd()           : Checking if CWD is already mounted: /home/chrisgor
DEBUG   [U=265085,P=24651] rootfs.c:64:singularity_rootfs_dir()       : Returning singularity_rootfs_dir: /var/singularity/mnt/final
DEBUG   [U=265085,P=24651] mount-util.c:42:check_mounted()            : Opening /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:48:check_mounted()            : Iterating through /proc/mounts
DEBUG   [U=265085,P=24651] mount-util.c:65:check_mounted()            : Mountpoint is already mounted: /home/chrisgor
VERBOSE [U=265085,P=24651] cwd.c:75:singularity_mount_cwd()           : Not mounting CWD (already mounted in container): /home/chrisgor
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
VERBOSE [U=0,P=24651]      rootfs.c:248:singularity_rootfs_chroot()   : Entering container file system root: /var/singularity/mnt/final
DEBUG   [U=0,P=24651]      privilege.c:179:singularity_priv_drop()    : Dropping privileges to UID=265085, GID=254778
DEBUG   [U=265085,P=24651] privilege.c:191:singularity_priv_drop()    : Confirming we have correct UID/GID
DEBUG   [U=265085,P=24651] rootfs.c:255:singularity_rootfs_chroot()   : Changing dir to '/' within the new root
DEBUG   [U=265085,P=24651] privilege.c:216:singularity_priv_drop_perm(): Called singularity_priv_drop_perm(void)
DEBUG   [U=265085,P=24651] privilege.c:233:singularity_priv_drop_perm(): Escalating permissison so we can properly drop permission
DEBUG   [U=265085,P=24651] privilege.c:152:singularity_priv_escalate(): Temporarily escalating privileges (U=265085)
DEBUG   [U=0,P=24651]      privilege.c:236:singularity_priv_drop_perm(): Resetting supplementary groups
DEBUG   [U=0,P=24651]      privilege.c:242:singularity_priv_drop_perm(): Dropping to group ID '254778'
DEBUG   [U=0,P=24651]      privilege.c:248:singularity_priv_drop_perm(): Dropping real and effective privileges to GID = '254778'
DEBUG   [U=0,P=24651]      privilege.c:254:singularity_priv_drop_perm(): Dropping real and effective privileges to UID = '265085'
DEBUG   [U=265085,P=24651] privilege.c:260:singularity_priv_drop_perm(): Confirming we have correct GID
DEBUG   [U=265085,P=24651] privilege.c:266:singularity_priv_drop_perm(): Confirming we have correct UID
DEBUG   [U=265085,P=24651] privilege.c:274:singularity_priv_drop_perm(): Setting NO_NEW_PRIVS to prevent future privilege escalations.
DEBUG   [U=265085,P=24651] privilege.c:284:singularity_priv_drop_perm(): Finished dropping privileges
DEBUG   [U=265085,P=24651] action.c:108:singularity_action_do()       : Trying to change directory to where we started
DEBUG   [U=265085,P=24651] util/util.c:102:envar_path()               : Checking environment variable is valid path: 'SINGULARITY_TARGET_PWD'
VERBOSE [U=265085,P=24651] util/util.c:50:envar()                     : Checking input from environment: 'SINGULARITY_TARGET_PWD'
DEBUG   [U=265085,P=24651] util/util.c:52:envar()                     : Checking environment variable is defined: SINGULARITY_TARGET_PWD
VERBOSE [U=265085,P=24651] util/util.c:54:envar()                     : Environment variable is NULL: SINGULARITY_TARGET_PWD
DEBUG   [U=265085,P=24651] action.c:136:singularity_action_do()       : Running action: shell
INFO    [U=265085,P=24651] shell.c:41:action_shell_do()               : Singularity: Invoking an interactive shell within container...

VERBOSE [U=265085,P=24651] shell.c:43:action_shell_do()               : Invoking the container's /.shell
DEBUG   [U=265085,P=24651] shell.c:45:action_shell_do()               : Found container's /.shell, executing that
bash: module: command not found
chrisgor@sherlock-ln02:~$
chrisgorgo picture chrisgorgo on 5 Jan 2017

Good call @vsoch, check out this debugging line:

VERBOSE [U=265085,P=24651] home.c:74:singularity_mount_home()         : Not mounting home directory (already mounted in container): /home/chrisgor
gmkurtzer picture gmkurtzer on 5 Jan 2017

Any more thoughts on the topic? I really think that if we do not change the defaults we will have a lot of confused users (especially in context of PYTHOPATH env var). I strongly vote for making the passing of the environment variables and mounting $HOME optional and OFF by default. It will save us a lot of headaches with user support.

chrisgorgo picture chrisgorgo on 4 Feb 2017

Changing semantics and behaviors for existing workflows is bad, bad, bad. Users hate that more than confusing semantics in the first place.

For example, _only_ changing the default would break the HTCondor integration.

I'd prefer to go the direction outlined above - change the default for new users but keep existing containers' behavior.

bbockelm picture bbockelm on 4 Feb 2017

I agree with @bbockelm, we can't change existing precedent, but this is a great idea. We have the -H option (which should work even if home directory mounts are being binded in the "development" branch). I did however make cleaning the environment much easier to use:

$ singularity shell -e /tmp/centos.img 
Singularity: Invoking an interactive shell within container...

Singularity centos.img> export
declare -x LD_LIBRARY_PATH=":/usr/local/lib:/usr/local/lib64"
declare -x LS_COLORS=""
declare -x OLDPWD
declare -x PATH="/usr/local/bin:/usr/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"
declare -x PS1="Singularity centos.img> "
declare -x PWD="/home/gmk/git/singularity"
declare -x SHLVL="1"
declare -x SINGULARITY_CONTAINER="centos.img"
declare -x SINGULARITY_INIT="1"
declare -x SINGULARITY_NAME="centos.img"
Singularity centos.img> exit

Note: The new -e/--cleanenv action verb options in the new "development" branch.

Note2: This feature is implemented in the C part of the execution path, which means that any envars set in /etc/singularity/init will also be "cleaned".

gmkurtzer picture gmkurtzer on 20 Mar 2017

Excited about the -e option!

chrisgorgo picture chrisgorgo on 21 Mar 2017

Between the -H option, and the -e options, I think this PR is satisfied so I am closing. Please comment and let me know if it needs to be reopened.

Thanks everyone!

gmkurtzer picture gmkurtzer on 25 Mar 2017
👍1

In my case, PYTHONPATH on the host had nothing set, but python inside my container was still searching per-user site-packages directories on the host (e.g. ~/.local/lib/pythonX.Y/site-packages), causing weird errors.

The solution was either

%runscript
    export PYTHONNOUSERSITE="some-arbitrary-value"
    python your-code.py

or

%runscript
    python -s your-code.py

-H wasn't an option for me because I still want to mount user's home directory.

hisplan picture hisplan on 13 Apr 2017

@hisplan This is a very good point and additionally I want to mention that in the 2.3 development branch you can also use the %environment section to do your export. Just make sure (for compatibility) you don't use Bash'isms (e.g. exporting on the same line as the variable declaration).

Thanks!

gmkurtzer picture gmkurtzer on 13 Apr 2017
👍1

%environment is just for the separation of concerns? If not, what's the difference between:

%runscript
    export PYTHONNOUSERSITE="some-arbitrary-value"
    python your-code.py

vs.

%environment
    export PYTHONNOUSERSITE="some-arbitrary-value"
%runscript
    python your-code.py
hisplan picture hisplan on 14 Apr 2017

In that case, the use of %environment will make PYTHONNOUSERSITE available via all Singularity action commands (e.g. shell, run, exec). If you put it only in the %runscript then it will only be available to the run command.

Also, I stress, that not all versions of /bin/sh (which is what gets called by the %runscript and everything else, will be linked to Bash. Thus defining the export on the same line of the variable may not always work. Standard Bourne syntax specifies/requires these being on two separate lines:

PYTHONNOUSERSITE="some-arbitrary-value"
export PYTHONNOUSERSITE

Sorry about being a nag about shell pedantics, but I've already seen this mistake bite others.

Hope that helps!

gmkurtzer picture gmkurtzer on 15 Apr 2017
👍1

By the way, it appears that the -e option is already implemented, thus the status of this issue is closed, but since it is not "officially" released (CMIW), it would be very helpful if you guys could use Git Issues' Milestone or Label to advertise some useful information to people like me searching for solutions, something like Milestone = v2.3, so that I don't have to keep asking "is this released?"

I appreciate all the hard work you guys put into this!

hisplan picture hisplan on 17 Apr 2017

Done! But will I remember for future issues? Hopefully, but I may need reminding. ;)

gmkurtzer picture gmkurtzer on 17 Apr 2017
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

mr-c picture mr-c  路  4Comments
alalazo picture alalazo  路  3Comments
Amir-Arsalan picture Amir-Arsalan  路  3Comments
dtrudg picture dtrudg  路  3Comments
DrDaveD picture DrDaveD  路  4Comments