Fmriprep: Publish to SingularityHub

[effigies]

It might be good to start having SingularityHub build images for released versions. I'm not sure how easy this is to sync with our Dockerfile.

[chrisgorgo]

Last time what stopped us was the inability to push images to Singularity Hub (the only option was for Singularity Hub to build images on the server). That lead to a bit of a chicken and egg situation during releases (if a minimal singularity recipy was used) or a risk for having different images in singularity hub and docker hub since they were built at different times and in different environments. Maybe those issues have been resolved in the meantime.

[effigies]

Gotcha.

[chrisgorgo]

All these imperfections/issues might be outweighed by the convenience of providing singularity images though - lets stay realistic.

[yarikoptic]

an explicit +1 on the issue -- would be nice to be able to just pull shub://
the problem with pulling from docker:// is inability to guarantee/verify that the image is the same since I guess upon regeneration would be a different checksum... thus -- no git-annex/datalad containers-add for me

[yarikoptic]

on the other hand, I thought: may be it is worth setting up

• bids-apps github organization
• automated script to monitor list of bids-apps/docker hub and generating per each bids-app repository which would provide a collection of Singularity.version files which would be merely e.g.

Bootstrap: docker
From: poldracklab/fmriprep:1.2.1@sha256:647a7901d4044aebeb8bc34351f346bc37482f52abd5ee4618964dc6e3df151e

with an explicit digest to guarantee that some rogue tag change would invalidate it ;-)

• and enabling singularity-hub to monitor all those repos...

This way it might be 1-to-1 correspondence with docker versions (given that bootstrapping from docker works as desired).

just an idea ;)

[effigies]

TBH I'm happy with any approach. I haven't looked into how to build Singularity images from recipes or push to singularityhub. If someone who does want to work on these things knows what they're doing, I'm content to let them go ahead. If it turns out to be suboptimal, we can try again in the future. But having a singularity image we can point to as canonical would be great.

[yarikoptic]

TBH I'm happy with any approach.

me too! ;)

[oesteban]

https://twitter.com/SingularityApp/status/1076502119811706880

[effigies]

In preparing for the potential for parallel distribution of Docker and Singularity images in FitLins, I've taken to maintaining a .neurodocker.json file. This file provides a single source that NeuroDocker can use to produce a Dockerfile and a Singularity recipe, which should help reduce the possibility of unsynchronized images.

If this seems like a good approach, a good first step would be to translate the current Dockerfile to a NeuroDocker description.

[oesteban]

Do you think neurodocker will be able to accommodate the idiosyncrasies of our docker image? (thinking of ICA-AROMA, svg/png compression and templateflow)

[effigies]

Sure, you can add arbitrary Docker commands like RUN. We can probably also add syntax to make node installations easier.

[oesteban]

Then it seems like a good idea to me.

[yarikoptic]

didn't know that there is a .json way to spec neurodocker. https://github.com/kaczmarj/neurodocker/tree/master/examples contain only generate.sh scripts so I just followed the suit.
FWIW, may be would give any idea, in my fight with the irreproducibility windmills, I am also trying to make containers themselves reproducible via ndfreeze which we added to neurodebian recently, and generating new versioned Singularityfiles so the hub has them all:
https://github.com/ReproNim/reproin/blob/master/generate_container.sh

[effigies]

https://github.com/kaczmarj/neurodocker/pull/229

And it's not completely trivial:

cat .neurodocker.json | \
    docker run --rm -i -a stdin -a stdout -a stderr \
    kaczmarj/neurodocker@sha256:23a13f8320f80b3c7e778e469cbc20847a3147c8f322b473f48ff2f66c5d1577 \
    generate docker - > Dockerfile

I actually generated the initial .neurodocker.json as follows:

<neurodocker command> --json | jq . > .neurodocker.json

I feel like it makes changes more diff-able to preserve the recipe in JSON. But we can go either way, really.

[effigies]

So apparently there's no Singularity equivalent to ARG. @kaczmarj Perhaps you have some suggestions for alternatives?

[kaczmarj]

@effigies - you are correct that there is no equivalent to ARG. you can pass environment variables at container build time, but it looks like the %labels section does not accept environment variables.

for example, if your singularity recipe looks like the one below and you have some build-time variables VERSION, you can build as follows:

(SINGULARITYENV_ docs here)

Bootstrap: docker
From: alpine
%post
  echo "$VERSION"
%labels
  version $VERSION

jakub@dash:~/tmp$ sudo SINGULARITYENV_VERSION=1.0.0 singularity build test.sif Singularity 
WARNING: Authentication token file not found : Only pulls of public images will succeed
INFO:    Starting build...
Getting image source signatures
Skipping fetch of repeat blob sha256:8e402f1a9c577ded051c1ef10e9fe4492890459522089959988a4852dee8ab2c
Copying config sha256:ff8df268d29ccbe81cdf0a173076dcfbbea4bb2b6df1dd26766a73cb7b4ae6f7
 585 B / 585 B [============================================================] 0s
Writing manifest to image destination
Storing signatures
INFO:    Running post scriptlet
+ echo 1.0.0
1.0.0
INFO:    Adding labels
INFO:    Creating SIF file...
INFO:    Build complete: test.sif

notice that in %post, $VERSION is resolved correctly. but it is not resolved in the %labels section. unfortunately, fmriprep likely wants the ARG functionality for the labels.

jakub@dash:~/tmp$ singularity inspect test.sif 

{
    "org.label-schema.build-date": "Tuesday_19_March_2019_14:44:41_EDT",
    "org.label-schema.schema-version": "1.0",
    "org.label-schema.usage.singularity.deffile.bootstrap": "docker",
    "org.label-schema.usage.singularity.deffile.from": "alpine",
    "org.label-schema.usage.singularity.version": "3.1.0",
    "version": "$VERSION"
}

i'm not sure how to get around this issue now, other than to set labels in the neurodocker command.

[rwblair]

Proof of concept that will trigger a singularity hub build immediately after a push to docker hub. The singularity built from the docker image with nothing else done in its definition file:
https://github.com/rwblair/webhook_test
https://hub.docker.com/r/rwblair/webhook_test
https://www.singularity-hub.org/collections/2871

Still need to parse the response from github when creating a deployment. If deployment creation (and thereby the singularity build trigger) fails circle will still show that step as successful.

[yarikoptic]

oh, sweet!
On my end I forgot to update that I also initiated https://github.com/ReproNim/containers which

• is a datalad dataset where I plan to "collect" all kinds of useful containers (ATM singularity but later docker too)
• for each bids app (for now I just do for selected set of bids apps) it provides Singularity file to have it built it from a Docker
• has a helper runner for singularity to guarantee isolated/reproducible execution with Singularity
• each one will have datalad-containers configuration established, so they would be
  
  
  
  • fetchable from Singularity hub storage
  
  
  • readily usable with datalad containers-run
  
  

See https://github.com/ReproNim/containers/issues for some list of planned TODOs etc. Would be nice to join efforts. In the least, I could just benefit from using "official" fmriprep singularity images, so to not torture singularity hub twice to build them ;)

Here I outlined a DataLad centric workflow I envision using/proposing

[oesteban]

@rwblair more reasons to push on this - https://neurostars.org/t/fmriprep-1-4-0-installation-issue/4333/2