Tools: Add parameters.settings.json to template and linting

[ewels]

Now that #241 is merged, we should add parameters.settings.json to the template and update the linting to check for this file and validate the JSON.

Note that whilst adding this it would be great to see if we can use the JSON file in nextflow code to reduce duplication:

• In the --help output
• In the summary log output at the start
• In nextflow.config

It would be great if we could end up with the basic parameters being _just_ set in parameters.settings.json

As a bonus, linting could also check the markdown documentation to ensure that all options are documented. We could also argue that usage documentation for the command line options is no longer required now that we have this metadata.

[phue]

I think this is a really cool feature, but I'm thinking that maybe we could discuss about doing something that @pditommaso already proposed here : https://github.com/nf-core/methylseq/pull/47

I think it would keep the pipeline scripts itself more readable if we move the functions (particularly readParamsFromJsonSettings(), formatParameterHelpData() and helpMessage()) to another file, as the code is the same for every pipeline.

What do you think @nf-core/all ?

[drpatelh]

I think this would be really cool too. Would remove alot of boilerplate code from main.nf. Fancy putting a PR together for tools @phue?

When we eventually port to using NF modules it seems we can bundle a similar file with all the other modules, and then import with an include statement!
https://github.com/nf-core/rnaseq/pull/162

[KochTobi]

@drpatelh there is already a PR (draft atm) containing the help message. I am currently a little short on time but feel free to modify it
https://github.com/nf-core/tools/pull/310

[drpatelh]

Ah I see. Sorry I missed that. Looking good!

Im happy to wait until you and @sven1103 are happy with it. Looks like it should just be a case of moving the functions to another file when you have the parameter parsing sorted.

Good luck getting the thesis together BTW 🏃 🤕 .

[phue]

I didn't even think about include :man_facepalming: that should actually be straightforward

I was playing around with implementing it in groovy directly similar to here https://github.com/nf-core/methylseq/pull/47 (but didn't succeed very far)

Edit:
Maybe we should wait a bit with this.. if I'm getting this right it would require to rewrite the pipelines in the workflow scope first

@pditommaso is there currently a way to include modules without setting nextflow.preview.dsl=2 ?

[pditommaso]

Sorry for the late reply. In the current version what it could be done is to create a library as a Groovy/Java class with static methods, eg

class MyLib {
  def foo(x, y) {
   """
   yout_command_script --x $x --y $y
   """
  }
}

Then you should be able to reference it from the pipeline script as MyLib.foo(1,2) (provided the module script is in the lib/ directory.

[phue]

I drafted something to the reference implementation of the JSON parsing:
https://github.com/nf-core/hlatyping/pull/61

Once this looks fine, we can port it over to the TEMPLATE :+1:

[ewels]

Linting for the new JSON Schema is now in place, moving the other items out to their own issues:

• Validate params in pipelines: https://github.com/nf-core/tools/issues/426
• Auto-build CLI help: https://github.com/nf-core/tools/issues/577
• Web help text: https://github.com/nf-core/nf-co.re/issues/393

Moving stuff out of nextflow.config isn't going to happen any time soon I think, so dropping that.

Much of the discussion on this thread is about moving boilerplate code into an external file. I've made a new issue for that here: https://github.com/nf-core/tools/issues/634

Now closing this thread, let me know if I missed anything.

Phil