Picongpu: Boost Error in ResourceLog

[BifengLei]

In combination with:

• GCC 7.3.1
• CUDA 9.2.88
• Boost 1.68.0 (C++11 build) OR Boost 1.65.1
• PIConGPU 0.4.0
• centos 7

I have checked the issue here:
https://github.com/ComputationalRadiationPhysics/picongpu/issues/2714#issue-367113446
But, there is still such err in ResourceLog.hpp:

/trinity/home/b.lei/pic/src/blei/picongpu/include/pmacc/../picongpu/plugins/ResourceLog.hpp(115): here
/trinity/shared/opt/rh/devtoolset-7/root/usr/include/c++/7/bits/stl_pair.h(331): error: identifier "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage" is undefined in device code

/trinity/shared/opt/rh/devtoolset-7/root/usr/include/c++/7/bits/stl_pair.h(331): error: identifier "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage" is undefined in device code

2 errors detected in the compilation of "/tmp/tmpxft_0000926d_00000000-6_main.cpp1.ii".
CMake Error at picongpu_generated_main.cpp.o.Release.cmake:279 (message):
  Error generating file
  /trinity/home/b.lei/pic/build/CMakeFiles/picongpu.dir//./picongpu_generated_main.cpp.o

[ax3l]

Thanks for reporting!

PIConGPU 0.4.0+ contains a first work-around from #2721 that should have solved the issue for boost::swap... But I see, that's indeed a regression! Now we still have a std::string signature lurking in device code.

@psychocoderHPC any idea? We definitely missed that the calling boost_swap_impl::swap has exactly the same std::string arguments we tried to get rid off ^^
Maybe we should just add a fat #ifndef __CUDA_ARCH__ guard in our ResourceLog::notify implementation to avoid any traces of boost::property_tree in device code.

[ax3l]

@BifengLei in the meantime, you can skip the compilation of the ResourceLog plugin altogether by commenting out these two lines:

diff --git a/include/picongpu/plugins/PluginController.hpp b/include/picongpu/plugins/PluginController.hpp
index ad6543d1c..e174c731f 100644
--- a/include/picongpu/plugins/PluginController.hpp
+++ b/include/picongpu/plugins/PluginController.hpp
@@ -70,7 +70,7 @@
 #endif

 #include "picongpu/plugins/Checkpoint.hpp"
-#include "picongpu/plugins/ResourceLog.hpp"
+// include "picongpu/plugins/ResourceLog.hpp"

 #include <pmacc/mappings/kernel/MappingDescription.hpp>

@@ -181,7 +181,7 @@ private:
 #if (ENABLE_HDF5 == 1)
         , plugins::multi::Master< hdf5::HDF5Writer >
 #endif
-        , ResourceLog
+//        , ResourceLog
     >;

[BifengLei]

@BifengLei in the meantime, you can skip the compilation of the ResourceLog plugin altogether by commenting out these two lines:

diff --git a/include/picongpu/plugins/PluginController.hpp b/include/picongpu/plugins/PluginController.hpp
index ad6543d1c..e174c731f 100644
--- a/include/picongpu/plugins/PluginController.hpp
+++ b/include/picongpu/plugins/PluginController.hpp
@@ -70,7 +70,7 @@
 #endif

 #include "picongpu/plugins/Checkpoint.hpp"
-#include "picongpu/plugins/ResourceLog.hpp"
+// include "picongpu/plugins/ResourceLog.hpp"

 #include <pmacc/mappings/kernel/MappingDescription.hpp>

@@ -181,7 +181,7 @@ private:
 #if (ENABLE_HDF5 == 1)
         , plugins::multi::Master< hdf5::HDF5Writer >
 #endif
-        , ResourceLog
+//        , ResourceLog
     >;

@ax3l Thank you so much for replying.
Unfortunately, there is still the err:

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/regex/v4/basic_regex_parser.hpp(48): warning: integer conversion resulted in a change of sign

/trinity/shared/opt/rh/devtoolset-7/root/usr/include/c++/7/bits/stl_pair.h(331): error: identifier "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage" is undefined in device code

/trinity/shared/opt/rh/devtoolset-7/root/usr/include/c++/7/bits/stl_pair.h(331): error: identifier "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage" is undefined in device code

2 errors detected in the compilation of "/tmp/tmpxft_00018152_00000000-6_main.cpp1.ii".
CMake Error at picongpu_generated_main.cpp.o.Release.cmake:279 (message):
  Error generating file
  /trinity/home/b.lei/pic/build/CMakeFiles/picongpu.dir//./picongpu_generated_main.cpp.o

[psychocoderHPC]

@psychocoderHPC any idea? We definitely missed that the calling boost_swap_impl::swap has exactly the same std::string arguments we tried to get rid off ^^
Maybe we should just add a fat #ifndef __CUDA_ARCH__ guard in our ResourceLog::notify implementation to avoid any traces of boost::property_tree in device code.

@ax3l yes we can try it

[ax3l]

@BifengLei can you post one line more regarding the new error? Which PIConGPU file includes the first line?

[BifengLei]

@ax3l Here is all of build information:

-- Found CUDA: /trinity/shared/opt/cuda-9.2 (found suitable version "9.2", minimum required is "7.0") 
-- ALPAKA_ACC_GPU_CUDA_ONLY_MODE
-- ALPAKA_ACC_GPU_CUDA_ENABLED
-- _cupla_COMPILE_OPTIONS_PUBLIC: -std=c++11;-ftemplate-depth-512;-Wall;-Wextra;-Wno-unknown-pragmas;-Wno-unused-parameter;-Wno-unused-local-typedefs;-Wno-attributes;-Wno-reorder;-Wno-sign-compare
-- _cupla_COMPILE_DEFINITIONS_PUBLIC: ALPAKA_ACC_GPU_CUDA_ONLY_MODE;ALPAKA_ACC_GPU_CUDA_ENABLED;ALPAKA_DEBUG=0;CUPLA_STREAM_ASYNC_ENABLED=1
-- _cupla_INCLUDE_DIRECTORIES_PUBLIC: /trinity/shared/opt/cuda-9.2/include;/trinity/home/b.lei/pic/src/blei/picongpu/thirdParty/alpaka/include;/trinity/home/b.lei/pic/src/blei/picongpu/thirdParty/cupla/include
-- _cupla_LINK_LIBRARIES_PUBLIC: Boost::boost;general;/trinity/shared/opt/cuda-9.2/lib64/libcudart.so;general;pthread;general;rt
-- Boost version: 1.68.0
-- Found the following Boost libraries:
--   filesystem
--   system
--   math_tr1
-- Boost: result_of with TR1 style and decltype fallback
-- Found OpenMP_C: -fopenmp  
-- Found OpenMP_CXX: -fopenmp  
-- Found OpenMP: TRUE   
-- Found CUDA: /trinity/shared/opt/cuda-9.2 (found suitable version "9.2", minimum required is "5.0") 
-- Boost version: 1.68.0
-- Found OpenMP_C: -fopenmp  
-- Found OpenMP_CXX: -fopenmp  
-- Boost version: 1.68.0
-- Boost version: 1.68.0
-- Found the following Boost libraries:
--   program_options
--   regex
--   filesystem
--   system
--   math_tr1
--   serialization
-- Found CUDA: /trinity/shared/opt/cuda-9.2 (found version "9.2") 
-- Could NOT find NVML (missing: NVML_LIBRARY) 
-- Boost version: 1.68.0
-- Found the following Boost libraries:
--   program_options
-- Can NOT find 'adios_config' - set ADIOS_ROOT or check your PATH
-- Could NOT find ADIOS (missing: ADIOS_LIBRARIES ADIOS_INCLUDE_DIRS) (Required is at least version "1.13.1")
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- Found Splash: /trinity/home/b.lei/lib/gcc-7.3.1/splash-1.68.0/lib/cmake/Splash
-- Found PNGwriter: /trinity/home/b.lei/lib/gcc-7.3.1/pngwriter/lib/cmake/PNGwriter
-- Could NOT find ISAAC - set ISAAC_DIR or check your CMAKE_PREFIX_PATH
-- Configuring done
-- Generating done
-- Build files have been written to: /trinity/home/b.lei/pic/build
[  4%] Built target cupla
[  8%] Linking CXX executable cuda_memtest
[ 20%] Built target mpiInfo
[ 50%] Built target picongpu-hostonly
/trinity/shared/opt/rh/devtoolset-7/root/usr/bin/../libexec/gcc/x86_64-redhat-linux/7/ld: warning: libmpi.so.12, needed by /trinity/home/b.lei/lib/gcc-7.3.1/zlib/lib/libz.so, may conflict with libmpi.so.40
[ 62%] Built target cuda_memtest
[ 66%] Building NVCC (Device) object CMakeFiles/picongpu.dir/picongpu_generated_main.cpp.o
/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/mpl/map/aux_/item.hpp(54): warning: "boost::mpl::aux::type_wrapper<T> operator/(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::aux::type_wrapper<Key> *)" declares a non-template function -- add <> to refer to a template instance

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/mpl/map/aux_/item.hpp(55): warning: "boost::mpl::aux::type_wrapper<boost::mpl::pair<Key, T>> operator|(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::next<Base::order>::type *)" declares a non-template function -- add <> to refer to a template instance

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/mpl/map/aux_/item.hpp(56): warning: "char (&operator||(const boost::mpl::m_item<Key, T, Base> &, boost::mpl::aux::type_wrapper<Key> *))[boost::mpl::next<Base::order>::type::value]" declares a non-template function -- add <> to refer to a template instance

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/mpl/map/aux_/item.hpp(71): warning: "boost::mpl::aux::type_wrapper<mpl_::void_> operator/(const boost::mpl::m_mask<Key, Base> &, boost::mpl::aux::type_wrapper<Key> *)" declares a non-template function -- add <> to refer to a template instance

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/mpl/map/aux_/item.hpp(72): warning: "boost::mpl::aux::type_wrapper<mpl_::void_> operator|(const boost::mpl::m_mask<Key, Base> &, boost::mpl::x_order_impl<Base, Key>::type *)" declares a non-template function -- add <> to refer to a template instance

/trinity/home/b.lei/pic/src/blei/picongpu/include/pmacc/../pmacc/particles/memory/dataTypes/Particle.hpp(167): warning: __device__ annotation is ignored on a function("operator=") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/pic/src/blei/picongpu/include/pmacc/../pmacc/particles/memory/dataTypes/Particle.hpp(167): warning: __host__ annotation is ignored on a function("operator=") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/pic/src/blei/picongpu/include/pmacc/../picongpu/fields/LaserPhysics.hpp(147): warning: pointless comparison of unsigned integer with zero

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/fusion/support/unused.hpp(25): warning: __host__ annotation is ignored on a function("unused_type") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/fusion/support/unused.hpp(25): warning: __device__ annotation is ignored on a function("unused_type") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/fusion/support/unused.hpp(31): warning: __host__ annotation is ignored on a function("unused_type") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/fusion/support/unused.hpp(31): warning: __device__ annotation is ignored on a function("unused_type") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/fusion/container/vector/vector.hpp(189): warning: __host__ annotation is ignored on a function("vector_data") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/fusion/container/vector/vector.hpp(189): warning: __device__ annotation is ignored on a function("vector_data") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/fusion/container/vector/vector.hpp(276): warning: __host__ annotation is ignored on a function("vector") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/fusion/container/vector/vector.hpp(276): warning: __device__ annotation is ignored on a function("vector") that is explicitly defaulted on its first declaration

/trinity/home/b.lei/lib/gcc-7.3.1/boost-1.68.0/include/boost/regex/v4/basic_regex_parser.hpp(48): warning: integer conversion resulted in a change of sign

/trinity/shared/opt/rh/devtoolset-7/root/usr/include/c++/7/bits/stl_pair.h(331): error: identifier "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage" is undefined in device code

/trinity/shared/opt/rh/devtoolset-7/root/usr/include/c++/7/bits/stl_pair.h(331): error: identifier "std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_empty_rep_storage" is undefined in device code

2 errors detected in the compilation of "/tmp/tmpxft_0005f9c5_00000000-6_main.cpp1.ii".
CMake Error at picongpu_generated_main.cpp.o.Release.cmake:279 (message):
  Error generating file
  /trinity/home/b.lei/pic/build/CMakeFiles/picongpu.dir//./picongpu_generated_main.cpp.o


make[2]: *** [CMakeFiles/picongpu.dir/picongpu_generated_main.cpp.o] Error 1
make[1]: *** [CMakeFiles/picongpu.dir/all] Error 2
make: *** [all] Error 2

[ax3l]

Hm, that's indeed all we got here...

I expect it's coming from include/picongpu/initialization/ParserGridDistribution.hpp where we can replace <boost/regex.hpp> with C++11 <regex> these days (GCC 4.9+ and the like).

Until then, and if you don't use the --gridDist feature for static load balancing, you can hack this out via:

diff --git a/include/picongpu/initialization/ParserGridDistribution.hpp b/include/picongpu/initialization/ParserGridDistribution.hpp
index e41a8cdc6..53c395952 100644
--- a/include/picongpu/initialization/ParserGridDistribution.hpp
+++ b/include/picongpu/initialization/ParserGridDistribution.hpp
@@ -24,7 +24,7 @@
 #include <string>   // std::string
 #include <iterator> // std::distance

-#include <boost/regex.hpp>
+// include <boost/regex.hpp>
 #include <boost/lexical_cast.hpp>


@@ -142,6 +142,9 @@ private:
     value_type
     parse( std::string const s ) const
     {
+        value_type newInput;
+
+#if 0
         boost::regex regFind( "[0-9]+(\\{[0-9]+\\})*",
                               boost::regex_constants::perl );

@@ -179,7 +182,7 @@ private:
             };
             newInput.emplace_back( g );
         }
-
+#endif
         return newInput;
     }

[BifengLei]

With gcc-4.9.2, not any err here.
But, with gcc-7.3.1 together with this temporal solution, the same err is still there.

[psychocoderHPC]

@ax3l If ParserGridDistribution is the reason than we can create a cpp file and interfacing to it via a *.hpp file.
I will open a PR for it.

[ax3l]

@BifengLei please be aware that GCC 7.3.1 is newer than what NVCC 9.2 "knew" (think: "could test against") at its release (May 2018).
GCC 7.3.0 was released on Jan 25, 2018 and might be the last reliably working version for that release of CUDA.

[BifengLei]

@ax3l Thanks a lot. I will try GCC7.3.0 or lower version later.

[ax3l]

@BifengLei this problem should be solved in our dev branch with #2899 .
Thanks @psychocoderHPC for the fix!