Nextflow: Add a directive that allows the fetching of tool version meta information

[pditommaso]

The goal of this proposal is to add a process directive that allow the user to specify one-liner command to be executed as very first execution to fetch the tool command version. For example:

process foo {
  container: {   }
   versionCmd: "bwa --version"

   input: 
   file x from z
   script:
   """
   do-this.sh
   """
}

The version command should be executed as the specified by the process configuration i.e. using the provided executor/container/module settings.

More than one versionCmd should be possible.

To avoid to spawn multiple jobs the versionCmd command should be prefixed to the real command script execution, only the first time. Moreover the version command should be trigger each time a dynamic container or module configuration is applied.

[pditommaso]

This may required #540

[pditommaso]

Also this version should be collected to include in the execution report and trace report. As possible extension it should be possible to collect these information using a channel and send to a reporting process (reads MultiQC).

[pditommaso]

Brainstormed a bit more with @emi80 about this. Since each task can potentially use a different container or run in a different host with a different env (when container are not used).

The proper way to handle this would be to executed the versionCmd before each task execution and capture the output in the command log using a special marker. Then NF should parse this log fetch the version info and track in the task execution metadata.

The versionCmd can be an arbitrary bash script that can chain one more command to properly dump the version of one or more tools used in the target task script.

[ewels]

Yes, I guess having it on task level is better. Shouldn’t be too tricky to collapse duplicates for MultiQC etc..

[pditommaso]

the new pic! 😆

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[ewels]

Bump

[pditommaso]

(sorry for the spam! 😄)

[ewels]

No problem! I'm not bumping all of these, I'm letting the ones I don't care about much die quietly 😅