Nextflow: join() behaves unexpectedly when no matches are found

[thpar]

Bug report

When joining a channel containing a list of pairs with an empty channel (setting remainder: true), the output is not simply the original channel as would be expected. It will only emit the keys instead.

Expected behavior and actual behavior

left = Channel.from(['X', 1], ['Y', 2], ['Z', 3], ['P', 7])
right= Channel.empty()
left.join(right, remainder: true).println()

should emit all unmatched pairs:

[Y, 2, null]
[Z, 3, null]
[X, 1, null]
[P, 7, null]

instead, only the keys are emitted:

X
Y
Z
P

Environment

• Nextflow version: 0.27.6 build 4775
• Java version: 1.8.0_60
• Operating system: Linux

Additional context

This gives unexpected errors when (because of a filter) the second channel stays empty, but the following channel operations still expect value pairs, not single values.

[thpar]

More generally: this behaviour seems to occur when not a single match is found.

left = Channel.from(['X', 1], ['Y', 2], ['Z', 3], ['P', 7])
right= Channel.from(['Q', 3], ['R', 7])

left.join(right, remainder: true).println()

Output:

X
Q
Y
R
Z
P

(replacing eg. Q in right by P would already change the output to the correct lists)

[pditommaso]

I've found the problem, tho not sure that the correct output should include null. More inclined to think that joining with an empty should should be the left channel itself, ie.

[Y, 2] 
[Z, 3] 
[X, 1] 
[P, 7]

[thpar]

But I'd say you always are expecting (in this example) triples as output.

Otherwise you'd have to catch downstream whether there was a match (eg. ['P', 7, 3]) or not (['P', 7]). The null would make it clear that it's an unmatched item and from which channel it came.

[pditommaso]

Yes, you are right.

[pditommaso]

Included in version 19.02.0-edge.

[rsuchecki]

Perfect timing, just hit upon this problem, and the fix is spot on!

:100:

Minor point, wile trying things out I had to use -ansi-log false to get stdout from pipeline script, which makes sense, just something I expect will come-up a lot. As ansi logging becomes default this will obviously need to be covered in docu, but perhaps more is needed, e.g. capturing/redirecting stdout to a file, printing a warning?

996

[pditommaso]

Nice. Do we want move discussion about ansi-log on #996?