Cudf: [BUG] Input validity presence checked instead of null_count

[jlowe]

Describe the bug
Some places in the code are checking for the presence of a validity buffer in the input column(s) instead of null_count to see whether the output needs a validity buffer allocated. For example, gather assumes the output column needs a validity buffer allocated if the input column has one, even if the input column's null_count is zero. Similarly, gdf_column_concat will assume the output column has a validity buffer allocated if any input vector has a validity buffer allocated, even if the null counts of all input columns are zero.

Expected behavior
Operations that can only produce null output values if at least one input column has null values should not require output columns to have an allocated validity buffer when the null_count of all input columns are zero.

[jrhemstad]

I think this problem will largely resolve itself as libcudf APIs are transitioned to _return_ outputs rather than requiring the user to pre-allocate outputs and pass via output paremeters.

In this way, the libcudf function will be responsible for allocating the output buffers.

See https://github.com/rapidsai/cudf/blob/branch-0.8/cpp/docs/TRANSITIONGUIDE.md

[jlowe]

I see a transition to returning outputs as somewhat tangential to this. If this gets fixed as part of that transition then that's great, but I don't see that as being expected. I would assume the transition would have libcudf allocate output buffers when it currently expects them to be allocated by the caller, and it seems to me some of those current expectations are incorrect.

[jrhemstad]

I agree that the functions you mentioned have incorrect expectations for the state of the output validity buffers.

I guess what I really wanted to say is that I don't expect these behaviors will be corrected until those APIs are updated.

[revans2]

I have run into this on gdf_unary and gdf_binary operations too.

[jrhemstad]

@devavret I believe you're currently refactoring the unary ops? Could you fix this while you're at it?

[jlowe]

I believe this problem has been resolved in the new libcudf++ code. Implementations are calling has_nulls on inputs to determine if outputs should have validity buffers allocated.