Cudf: [BUG] src/join/join_kernels.cuh(406): error: no instance of function template "cub::ShuffleIndex" matches the argument list

[MasoodK]

Hi guys,

Describe the bug
I got the same following error when compiling both release v0.4.0 and master (as of Jan 26):
/home/dev/cudf_dev/cudf-0.4.0/cpp/src/join/join_kernels.cuh(406): error: no instance of function template "cub::ShuffleIndex" matches the argument list
argument types are: (int64_t, int, enum mgpu::, const unsigned int)
detected during:
instantiation of "void probe_hash_table(const multimap_type *, const gdf_table &, const gdf_table &, size_type, output_index_type *, output_index_type *, size_type *, size_type, __nv_bool, output_index_type) [with join_type=JoinType::LEFT_JOIN, multimap_type=concurrent_unordered_multimap, equal_to, legacy_allocator>, false>, key_type=hash_value_type, size_type=int64_t, output_index_type=output_index_type, block_size=128L, output_cache_size=128L]"

Steps/Code to reproduce bug
Building libcudf and at make -j

Environment details:

• Environment location: AWS p3.2xlarge (V100), CentOS
• Method of cuDF install: from source
• Please run and attach the output of the cudf/print_env.sh script to gather relevant environment details: https://gist.github.com/MasoodK/2c697fc6cec02f6a2656e1d4fb98a28c

Additional context

I understand that the compile/build is being tested for Ubuntu, but looks like someone has managed to make it work in CentOS (#688), but I am not sure how to go about that and not mess up the current setup; I just spent hours upgrading the OS tools to gcc 7.3, etc., and it's currently stuck at the above compile error. I am filing this issue as bug, as I have followed the detailed env setup and ran into this compile error. If you believe this is specific to CentOS or my setup, I greatly appreciate some more documentation/instructions. Thanks!

[datametrician]

Thank you for the detailed bug. We’re working on CentOS support actively. Someone will look into this soon.

[jrhemstad]

This bug only occurs if you're using a newer version of CUB than we support.

Our CUB submodule is pinned to an older commit than the latest version. There were breaking changes in their API that we haven't updated yet, which is why we don't use the latest CUB.

[MasoodK]

Thanks @jrhemstad for the hint; yes, I had fetched the thirdparty modules separately and from their master. I got them at the mentioned commits and it got me passed the above error, however, I am now hitting these errors:

1) Compiling branch v0.6:
libNVStrings.so: undefined reference to `std::runtime_error::runtime_error(char const*)@GLIBCXX_3.4.21'

Full output of make -j: https://gist.github.com/MasoodK/6da0cd4beda38dd390f14fa8b8893dfb

2) Compiling release 0.4.0:
cpp/tests/types/types_test.cpp:42:8: error: ‘mt19937’ in namespace ‘std’ does not name a type std::mt19937 rng

Full output of make -j: https://gist.github.com/MasoodK/3c4b5777d482bb16ae979231da6b8f86

print_env.sh output: https://gist.github.com/MasoodK/88024570f67a020e747fdf13fb846e58

@datametrician, let me know if you need me to create new issues (if so, one for each of the above two errors?)

Thanks

[jrhemstad]

1. Is a real issue with CentOS support in NVStrings that we haven't resolved
2. Is a missing include that was fixed in 0.5

[MasoodK]

OK, thanks @jrhemstad for the response. I gather then that the path to compile this lib on CentOS is effectively blocked for the time being, unless someone can provide more instructions regarding the conda solution mentioned in #688; I'll ask there.

[jrhemstad]

@MasoodK yeah, I believe we haven't done the work to officially support CentOS yet.

As a pointer in the direction of a possible solution, you can look into the CentOS Developer Toolset repositories that provide current versions of gcc and other build tools.

It would be very helpful if you could create a feature request on GitHub asking for CentOS support.

Otherwise I'm closing this issue as I believe your CUB problem has been resolved.