Cudf: [BUG] Java aggregation tests are crashing the JVM
Describe the bug
After #6392 some groupby aggregation tests in TableTest are failing. One in particular is testGroupByAvg which tests mean aggregation. The JVM crashes hard with no Java stacktrace. Instead it terminates the process with this on stderr:
terminate called after throwing an instance of 'cudf::logic_error'
what(): cuDF failure at: ../include/cudf/detail/aggregation/aggregation.cuh:436: Unsupported aggregation for initializing values
The odd thing is we wrap all libcudf calls with catch (const std::exception& which somehow is not catching this error and providing a stacktrace.
Steps/Code to reproduce bug
In the java directory execute mvn test -Dtest=TableTest#testGroupByAvg
Expected behavior
Aggregations do not crash
jlowe
>All comments
Finally tracked this down why the Java aggregation tests are failing, it was a bit nasty. The problem occurs because the JNI code uses clone() on the aggregation instances. mean_aggregation derives from aggregation but does not override the clone() method. Therefore when the aggregation is cloned, it's copied as if it were a normal base aggregation and the specializiation method overrides are lost. When the cloned aggregation attempts to generate the simple aggregations it just returns itself as a simple aggregation due to the lack of override, causing a crash later.
jlowe
on 4 Dec 2020
Related issues
randerzander
路
3Comments
ericmjl
路
3Comments
stevencarlislewalker
路
3Comments
razajafri
路
3Comments
saifrahmed
路
3Comments
Most helpful comment
Finally tracked this down why the Java aggregation tests are failing, it was a bit nasty. The problem occurs because the JNI code uses
clone()on the aggregation instances.mean_aggregationderives fromaggregationbut does not override theclone()method. Therefore when the aggregation is cloned, it's copied as if it were a normal base aggregation and the specializiation method overrides are lost. When the cloned aggregation attempts to generate the simple aggregations it just returns itself as a simple aggregation due to the lack of override, causing a crash later.