Singularity: failed to pull image, mksquashfs: executable file not found in $PATH

[pillepalle123]

Version of Singularity:

3.0.2-96.gd61d61e2

Expected behavior

Pulling singularity image docker://mono:5.12.0.226

I am sorry if this doesn't belong here. I am having issues with pulling a singularity image.
I can not use root since it has to run on a cluster.

thanks a lot!

Actual behavior

/UltraQuant-master> snakemake --snakefile UltraQuant.sm --configfile config.yaml --use-singularity
Building DAG of jobs...
Pulling singularity image docker://mono:5.12.0.226.
WorkflowError:
Failed to pull singularity image from docker://mono:5.12.0.226:
WARNING: Authentication token file not found : Only pulls of public images will succeed
INFO:    Starting build...
Getting image source signatures
Skipping fetch of repeat blob sha256:57936531d1eea907ae6c73ebe8f8b5dc71232f5a642db22e877a4f0fc6ff1516
Skipping fetch of repeat blob sha256:8b5f010d42dee2116f29314cb00c4359ddc86cfaa48e8ab7b516fe4e6d85409a
Skipping fetch of repeat blob sha256:a24afbee490d043be22ea64d08d3b307a7c6ef8d3a9524c40f1077809a22a540
Skipping fetch of repeat blob sha256:3d4b2fe5ed77bc31c1704eb721730de6cdd7646b05c2caa478dfdc1a60bc457c
Copying config sha256:b98714621ec1db52300dcb037c7aaf86c0a19af379004ddd7e2cb1249f8b5e2b
 1.71 KiB / 1.71 KiB [======================================================] 0s
Writing manifest to image destination
Storing signatures
INFO:    Creating SIF file...
FATAL:   Unable to pull docker://mono:5.12.0.226: While searching for mksquashfs: exec: "mksquashfs": executable file not found in $PATH

Steps to reproduce behavior

install UltraQuant and run:

/UltraQuant-master> snakemake --snakefile UltraQuant.sm --configfile config.yaml --use-singularity

[kaczmarj]

@pillepalle123 - mksquashfs is provided by the system package squashfs-tools. others should correct me, but I'm fairly certain you need squashfs-tools to run singularity properly. you can ask your cluster administrators to install that package.

[jmstover]

You don't need mksquashfs (the squashfs-tools package) to run Singularity, but you do need it on the system where you build images.

Squashfs support is needed in the Kernel where you're running images, but not the userspace squashfs-tools.

[pillepalle123]

@jmstover @kaczmarj great, thanks a lot! So would it be possible to install squashfs-tools in my home folder without bothering our admin all the time? If yes, how would that work? I have downloaded the .rpm and .ymp for suse SLE but having trouble installing it.

[kaczmarj]

@pillepalle123 - you can build squashfs-tools from source (https://github.com/plougher/squashfs-tools) and install into your home directory.

you could also extract the contents of the rpm somewhere in your home directory, but I would suggest building from source.

either way, you will also have to modify your $PATH variable to include whichever directory contains mksquashfs.

[pillepalle123]

@jmstover @kaczmarj I installed it and added it to my Path:

~> mksquashfs -version
mksquashfs version 4.3-git (2017/11/29)
copyright (C) 2017 Phillip Lougher <[email protected]>

But, I am still getting the same error... how can that be?

INFO:    Creating SIF file...
FATAL:   Unable to pull docker://mono:5.18.0.225: While searching for mksquashfs: exec: "mksquashfs": executable file not found in $PATH

is it correct that I have only two files (mksquashfs and unsquashfs) in my installation folder?

thx!

[kaczmarj]

@pillepalle123 - I think it's correct to only have those two executables in the installation folder. That's strange behavior. I'm not sure why singularity is not finding mksquashfs.

It looks like singularity expects mksquashfs to be in a certain set of directories, not necessarily in $PATH (source). However, to install into those directories (or to change the singularity config file), you need root privileges. Also, the error you are receiving suggests that singularity is trying to look in $PATH...

If it's any help, here are the lines that get the path to mksquashfs, and your error seems to be coming from this line.

[pillepalle123]

@kaczmarj thanks so much for figuring this out!
in both lines, it says: "(ignoring $PATH)"

exec.LookPath functions on absolute paths (ignoring $PATH) as well
is that maybe the reason it cannot find it?

[kaczmarj]

@pillepalle123 - Possibly, but because the error exec: "mksquashfs": executable file not found in $PATH is a golang error that indicates it is searching for mksquashfs. that doesn't contain slashes, so it should look in $PATH. I wonder if the program is looking in the $PATH of a different user or different process. I'm not sure.

edit: by different user, I mean maybe the root user's path for whatever reason. I'm really not sure!

[pillepalle123]

@kaczmarj thanks a lot! My admin installed squashfs-tools as root and now it is working.

[jscook2345]

Please let us know if you have any further questions @pillepalle123 !