Tools: workflow_summary_mqc.yaml is staged in the root of workDir

[jinmingda]

In the template generated by nf-core create, I noticed that the workflow summary workflow_summary_mqc.yaml will always be staged in the root of workDir. This is not a problem on a local machine. But in a distributed system (i.e. AWS Batch), where workDir can be shared by multiple nextflow runs at the same time, this might potentially cause file conflict. Here's a workaround I came up with to avoid this problem.

Channel.from(summary.collect{ [it.key, it.value] })
    .map { k,v -> "<dt>$k</dt><dd><samp>${v ?: '<span style=\"color:#999999;\">N/A</a>'}</samp></dd>" }
    .reduce { a, b -> return [a, b].join("\n            ") }
    .map { x -> """
    id: 'nf-core-mypipeline-summary'
    description: " - this information is collected when the pipeline is started."
    section_name: 'nf-core/mypipeline Workflow Summary'
    section_href: 'https://github.com/nf-core/mypipeline'
    plot_type: 'html'
    data: |
        <dl class=\"dl-horizontal\">
            $x
        </dl>
    """.stripIndent() }
    .set { ch_workflow_summary }

In the multiqc process, simply collect the YAML string into a file:

file workflow_summary from ch_workflow_summary.collectFile(name: "workflow_summary_mqc.yaml")

[ewels]

Interesting! I would have assumed that the file was created within the MultiQC work directory, as that's where the existing function is called..

Thanks for the fix - if you're able to submit it as a pull-request then it will get sorted faster 😃

[apeltzer]

Can confirm after a quick chekck that its indeed in the root dir of the work directory... So could potentially be problematic, though within a single pipeline run there won't be conflicts I suppose? Of course (as you mentioned) it could be problematic when running multiple pipelines simultaneously using the same work directory... Please do a PR :-)

[jinmingda]

@apeltzer @ewels I just submitted a pull request #490 to address this issue. Please review when you have time. Thank you!

[jinmingda]

The issue has been addressed in PR #490. Please feel free to re-open if there's any question.