Tools: Dynamic template synchronisation

[drpatelh]

I wonder if it's possible for pipelines to have a more dynamic synchronisation relative to the template code. This would then be something we can perform as part of the CI checks to ensure agreement with the template, and this would then also reduce the number of files that have to be reviewed when submitting PRs! Also, it will ensure that all pipelines are always in sync with the template via (possibly) smaller incremental changes rather than those needed when tools is released.

Using this approach we would essentially shift the burden of reviewing and suggesting updates to template specific files to the developer(s) of the pipeline and not the person performing the code review. Given that these updates will have to be performed at some point anyway!

Discuss...

[MaxUlysse]

I like this idea.
I would also like to have versions for the template that could be visible at the pipeline level.

[ewels]

It should be possible to do this and I'm hoping we can do it as part of the new nf-core/modules DSLv2. Loading boilerplate code in from a central repo (as a submodule with a fixed and stable hash).

[ewels]

To follow up - with the current setup, the only way we can make small-and-often updates in this way would be to sync with the dev version of the template. I don't think this really works, as the template is meant to be pretty stable etc.

Aside from using remotely loaded boilerplate code, I think that the best thing we can do is to do much more frequent releases of tools.

[drpatelh]

Yeah, I agree. It was wishful thinking. With regard to the template, the code we will be shifting to nf-core/modules will be utility functions etc so wont be that much compared to the entire template code base. Frequent releases of tools is the better solution if we can make it happen.