Picongpu: CUDA 9.2: Boost PropTree Error in ResourceLog

[ax3l]

In the combination of

• GCC 4.9.4
• CUDA 9.2.88
• Boost 1.65.1 (both C++11 build and C++98 "default" build)
• PIConGPU 0.4.0-rc2
• openSUSE Linux Workstation

@ajitup73 in #2704 sees the following compile error in our ResourceLog plugin:

/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.9.4/boost-1.65.1-dy5wuhsxhwb6imd6maxwaeo2jd7ppfsr/include/boost/core/swap.hpp(36): warning: calling a __host__ function from a __host__ __device__ function is not allowed
          detected during:
            instantiation of "void boost_swap_impl::swap_impl(T &, T &) [with T=std::string]" 
(56): here
            instantiation of "void boost::swap(T1 &, T2 &) [with T1=std::string, T2=std::string]" 
/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.9.4/boost-1.65.1-dy5wuhsxhwb6imd6maxwaeo2jd7ppfsr/include/boost/property_tree/detail/ptree_implementation.hpp(214): here
            instantiation of "void boost::property_tree::basic_ptree<Key, Data, KeyCompare>::swap(boost::property_tree::basic_ptree<Key, Data, KeyCompare>::self_type &) [with Key=std::string, Data=std::string, KeyCompare=std::less<std::string>]" 
/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.9.4/boost-1.65.1-dy5wuhsxhwb6imd6maxwaeo2jd7ppfsr/include/boost/property_tree/detail/ptree_implementation.hpp(201): here
            instantiation of "boost::property_tree::basic_ptree<Key, Data, KeyCompare>::self_type &boost::property_tree::basic_ptree<Key, Data, KeyCompare>::operator=(const boost::property_tree::basic_ptree<Key, Data, KeyCompare>::self_type &) [with Key=std::string, Data=std::string, KeyCompare=std::less<std::string>]" 
/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.9.4/boost-1.65.1-dy5wuhsxhwb6imd6maxwaeo2jd7ppfsr/include/boost/property_tree/detail/ptree_implementation.hpp(644): here
            instantiation of "boost::property_tree::basic_ptree<Key, Data, KeyCompare>::self_type &boost::property_tree::basic_ptree<Key, Data, KeyCompare>::put_child(const boost::property_tree::basic_ptree<Key, Data, KeyCompare>::path_type &, const boost::property_tree::basic_ptree<Key, Data, KeyCompare>::self_type &) [with Key=std::string, Data=std::string, KeyCompare=std::less<std::string>]" 
/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.9.4/boost-1.65.1-dy5wuhsxhwb6imd6maxwaeo2jd7ppfsr/include/boost/property_tree/detail/ptree_implementation.hpp(848): here
            instantiation of "boost::property_tree::basic_ptree<Key, Data, KeyCompare>::self_type &boost::property_tree::basic_ptree<Key, Data, KeyCompare>::put(const boost::property_tree::basic_ptree<Key, Data, KeyCompare>::path_type &, const Type &, Translator) [with Key=std::string, Data=std::string, KeyCompare=std::less<std::string>, Type=size_t, Translator=boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, size_t>]" 
/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.9.4/boost-1.65.1-dy5wuhsxhwb6imd6maxwaeo2jd7ppfsr/include/boost/property_tree/detail/ptree_implementation.hpp(860): here
            instantiation of "boost::property_tree::basic_ptree<Key, Data, KeyCompare>::self_type &boost::property_tree::basic_ptree<Key, Data, KeyCompare>::put(const boost::property_tree::basic_ptree<Key, Data, KeyCompare>::path_type &, const Type &) [with Key=std::string, Data=std::string, KeyCompare=std::less<std::string>, Type=size_t]" 
/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.9.4/picongpu-0.4.0-rc2-qunulnqkeboxj4w4uhzltzaqvak2frwq/include/pmacc/../picongpu/plugins/ResourceLog.hpp(95): here

/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.8/gcc-4.9.4-ta2pzw4p2wr62erizilz2uu3s2gdtleg/include/c++/4.9.4/bits/stl_pair.h(145): error: identifier "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage" is undefined in device code

/home/tempo/src/spack/opt/spack/linux-opensuse42-x86_64/gcc-4.8/gcc-4.9.4-ta2pzw4p2wr62erizilz2uu3s2gdtleg/include/c++/4.9.4/bits/stl_pair.h(145): error: identifier "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage" is undefined in device code

Have we seen this before? It looks to me, that somehow a std::string signature slips into device code. Maybe via default ctors in boost functionality for property trees inside the otherwise host-only notify?

Full outputs:

• pic-configure / cmake (ok)
• pic-build / make -j (fails with error above)

[ax3l]

(unconfirmed) -> (confirmed) Could be Not a problem with a C++98 boost build (GCC 4.9 default). I changed the spack recipe now to make sure we build the right boost C++11 variant to start with.

[ax3l]

I wonder if later releases of boost, such as 1.67.0, have the same problem still. If we are lucky we just need a later version.

[psychocoderHPC]

If we define an own swap within the boost_swap_impl namespace for string we can maybe solve the issue

namespace boost_swap_impl
{
    BOOST_GPU_ENABLED
    void swap(std::string& s1, std::string& s2)
    {
#ifndef __CUDA_ARCH__
        std::swap(left,right);
#endif
    }
}

[ax3l]

Sounds good, can you also provide an upstream fix for Boost.Core https://github.com/boostorg/core/blob/boost-1.68.0/include/boost/core/swap.hpp#L31-L37 please ?

[ax3l]

Trying to create a reproducer with NVCC 9.2:

docker pull nvidia/cuda:9.2-devel-ubuntu18.04
docker run -it nvidia/cuda:9.2-devel-ubuntu18.04

# in docker now
apt-get update
# boost 1.65.1, cmake 3.10.2, GNU 7.3.0
apt-get install -y git ca-certificates curl libboost-all-dev rsync cmake
cd /usr/local
git clone --branch release-0.4.0 https://github.com/ComputationalRadiationPhysics/picongpu.git
picongpu/bin/pic-create picongpu/share/picongpu/examples/LaserWakefield lwfa-dev
cd lwfa-dev
../picongpu/bin/pic-build -b cuda:30

This does not fail for me!

[ax3l]

Actually, I am not sure we should support this combination of (an ancient) GCC 4.9 and CUDA 9.2 anymore.
@ajitup73 the best to do is to use a recent, supported GCC such as the default GCC 7.3 with CUDA 9.2. I updated the install instruction in #2720 to reflect better combinations.

[ajitup73]

I did try gcc-7.3.0. However, the gcc compiler installation fails in spack. It gives error at some stage that isl is not found, whereas isl is already installed in the spack. A repeat install of gcc in spack shows that isl is installed. However, when it reaches to the step of gcc, it fails giving this error. I tried with gcc-8.1.0 and again it gives same error.

[ax3l]

Can you show me the error when installing GCC 7.3 please?
I checked the spack recipe and the selected version range might be off, I can fix that in spack itself.

[ajitup73]

I did not save the error. I will do it again and post it here.

[ax3l]

should be fixed with #2721