Nextflow: Module registry (DSL2)

[grst]

New feature

DSL2 enables the inclusion of modules, which is a great improvement.
I was wondering if we could go one step further and think of a public "module registry". Think of it as "PyPI for nextflow". I know you turned down "remote modules" in https://github.com/nextflow-io/nextflow/issues/1376, but a registry would be different in that it

• it would be versioned,
• stored at a central place, ensuring that published modules are not arbitrarily removed,
• would ensure consistent documentation of inputs/outputs of the modules.

This is all about re-using code and avoiding code duplication.

Usage scenario

I want to build a new pipeline that re-uses some standard stuff (e.g. RNAseq alignment) and then does sth custom.

Instead of re-implementing (or copying over and thereby duplicating) a process running STAR I could just, e,g.

include 'star:0.1'

and then use the process/workflow in the pipeline

star(fastq_channel)

Suggest implementation

The basic building blocks would be:

• A database to store modules
• A website to browse modules
• An API nextflow uses to download modules
• A command-line tool to push modules

I am aware that this is a project by itself and probably out-of-scope for the time being, but I think it's worth keeping in mind.

[grst]

Something in that direction is being discussed within the context of nf-core: https://github.com/nf-core/modules.

In particular this issue: https://github.com/nf-core/modules/issues/8

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.