Singularity: action files (shell) written twice, doesn't honor user changes

[vsoch]

Version of Singularity:

3.0 and later.

Expected behavior

I would expect to be able to write a custom message (or otherwise customize) the /.singularity.d/actions scripts (shell, for example) and have the change persist.

Actual behavior

The change does not persist. Here we see adding lines to the file during a build, and then they are not present after. I will provide full code to reproduce. Here is the recipe file:

Bootstrap: docker
From: ubuntu:16.04

# sudo singularity build greeting.sif Singularity

%post

    # Write an echo statement to the second and third line of the shell action
    sed -i '2iecho Hello, ${USER}!;' /.singularity.d/actions/shell
    sed -i '3iecho;' /.singularity.d/actions/shell
    cat /.singularity.d/actions/shell

and then build

$ sudo singularity build greeting.sif Singularity

and then we can see during the build that the top lines are added, as I expected (the "hello $USER" bit:)

...
#!/bin/sh
echo Hello, ${USER}!;
echo;

for script in /.singularity.d/env/*.sh; do
    if [ -f "$script" ]; then
        . "$script"
    fi
done

if test -n "$SINGULARITY_SHELL" -a -x "$SINGULARITY_SHELL"; then
    exec $SINGULARITY_SHELL "$@"

    echo "ERROR: Failed running shell as defined by '\$SINGULARITY_SHELL'" 1>&2
    exit 1

elif test -x /bin/bash; then
    SHELL=/bin/bash
    PS1="Singularity $SINGULARITY_NAME:\\w> "
    export SHELL PS1
    exec /bin/bash --norc "$@"
elif test -x /bin/sh; then
    SHELL=/bin/sh
    export SHELL
    exec /bin/sh "$@"
else
    echo "ERROR: /bin/sh does not exist in container" 1>&2
fi
exit 1
...

but then shelling inside, the file is "reset,"

$ singularity shell greeting.sif 
Singularity greeting.sif:~/Documents/Dropbox/Code/singularity/motd/greeting> cat /.singularity.d/actions/shell 
#!/bin/sh

for script in /.singularity.d/env/*.sh; do
    if [ -f "$script" ]; then
        . "$script"
    fi
done

if test -n "$SINGULARITY_SHELL" -a -x "$SINGULARITY_SHELL"; then
    exec $SINGULARITY_SHELL "$@"

    echo "ERROR: Failed running shell as defined by '\$SINGULARITY_SHELL'" 1>&2
    exit 1

elif test -x /bin/bash; then
    SHELL=/bin/bash
    PS1="Singularity $SINGULARITY_NAME:\\w> "
    export SHELL PS1
    exec /bin/bash --norc "$@"
elif test -x /bin/sh; then
    SHELL=/bin/sh
    export SHELL
    exec /bin/sh "$@"
else
    echo "ERROR: /bin/sh does not exist in container" 1>&2
fi
exit 1

I suspect Singularity is copying it twice, before and after, and not honoring user customization. As a use case, a user might want to add a message of the day or similar. The issue is reported first here https://github.com/singularityhub/motd/issues/2

Thanks for your help!

[GodloveD]

Afaik, one of the changes that was made in Singularity 3.x is to bind the action scripts directly into the container at runtime. The idea is that the action scripts can really be considered code and as such should not be part of the container itself. This change confers several advantages. For instance, if ever there were a bug discovered in the action scripts, it should be possible to update Singularity and make sure that update is applied to older containers as well. These files are not really meant to be manipulated by the user, so I don't think there's going to be a big desire to fix. @ikaneshiro may be able to provide more info.

[vsoch]

So - the files are generated on the fly during runtime, and the ones that are "existing" during the build are mistakingly there? @ikaneshiro I think (at least for shell) it would be wanted to be able to customize this file, for example to give the user a meaningful message when they shell in. I can understand the other actions aren't changeable (run comes from %runscript, and exec is set as well) but it seems like a reasonable use case to want to customize something for shell. How can we accomplish this?

[Trophime]

Like @vsoch I would like to add a message for the users when they starts singularity.
This is really a plus. Hope this will be fixed in 3.xx

[cclerget]

@Trophime @vsoch As all scripts source .sh files in /.singularity.d/env/, you can just add whatever you want in this directory, so you can fit your needs from a definition file with :

bootstrap: docker
from: busybox

%post
    cat > /.singularity.d/env/99-motd.sh <<EOF
case \$0 in
    /.singularity.d/actions/shell)
        echo "Hello \$USER from shell" ;;
    /.singularity.d/actions/exec)
        echo "Hello \$USER from exec" ;;
    /.singularity.d/actions/run)
        echo "Hello \$USER from run" ;;
    /.singularity.d/actions/test)
        echo "Hello \$USER from test" ;;
esac
EOF

[vsoch]

Ah, fantastic! @Trophime I’m traveling this morning but I’ll test out this method soon, and if it works for previous versions of Singularity I’ll update all the recipes for the motd repo. @cclerget as always, thank you kindly for your wisdom.

[vsoch]

Thanks for everyone's help! I've updated all the recipes to use this new strategy, and added a new "general" folder with a recipe that demonstrates what @cclerget showed above. It's even cooler than before! https://github.com/singularityhub/motd Thanks again, closing.