Picongpu: ICC: libPMacc support

[ax3l]

With the new taurus config (#622), it was a bit hard to find a corresponding cuda/intel/boost combination but I found a very recent one.

• intel/2013-sp1 (aka intel 14.0)
• boost/1.57.0-intel2013-sp1
• cuda/6.5.14

This pull already fixes a problem in src/libPMacc/include/math/Tuple.hpp where mpl::integral_c from <boost/mpl/integral_c.hpp> was missing an include #627. This pull is a follow-up to re-provide ICC support.

It seems to be in src/libPMacc/include/math/MapTuple.hpp:

[ 16%] Building NVCC (Device) object build_picongpu/CMakeFiles/picongpu.dir//./picongpu
_generated_main.cu.o
$PICSRC/src/libPMacc/include/math/MapTuple.hpp(107): error: incomplete type is not allowed
          detected during:
            instantiation of class "PMacc::math::MapTuple<Map_, PODType, false> [with M
ap_=boost::mpl::m_mask<picongpu::placeholder_definition15::momentum, boost::mpl::m_mask
<picongpu::placeholder_definition12::position<picongpu::placeholder_definition14::posit
ion_pic, PMacc::placeholder_definition11::pmacc_isAlias>, boost::mpl::m_item<5L, PMacc:
:placeholder_definition8::localCellIdx, PMacc::StaticArray<PMacc::placeholder_definitio
n8::localCellIdx::type, boost::mpl::integral_c<uint32_t, 1U>>, boost::mpl::m_item<4L, p
icongpu::placeholder_definition16::momentumPrev1, PMacc::StaticArray<picongpu::placehol
der_definition14::position_pic::type, boost::mpl::integral_c<uint32_t, 1U>>, boost::mpl
::m_item<3L, picongpu::placeholder_definition17::weighting, PMacc::StaticArray<picongpu
::placeholder_definition17::weighting::type, boost::mpl::integral_c<uint32_t, 1U>>, boo
st::mpl::m_item<2L, picongpu::placeholder_definition15::momentum, PMacc::StaticArray<pi
congpu::placeholder_definition14::position_pic::type, boost::mpl::integral_c<uint32_t, 
1U>>, boost::mpl::m_item<1L, picongpu::placeholder_definition12::position<picongpu::pla
ceholder_definition14::position_pic, PMacc::placeholder_definition11::pmacc_isAlias>, P
Macc::StaticArray<picongpu::placeholder_definition14::position_pic::type, boost::mpl::i
ntegral_c<uint32_t, 1U>>, boost::mpl::map<boost::mpl::na, boost::mpl::na, boost::mpl::n
a, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boos
t::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl:
:na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, bo
ost::mpl::na>>>>>>>>, PODType=PMacc::math::AlignedData]" 
(107): here
            instantiation of class "PMacc::math::MapTuple<Map_, PODType, false> [with M
ap_=boost::mpl::m_item<5L, PMacc::placeholder_definition8::localCellIdx, PMacc::StaticA
rray<PMacc::placeholder_definition8::localCellIdx::type, boost::mpl::integral_c<uint32_
t, 1U>>, boost::mpl::m_item<4L, picongpu::placeholder_definition16::momentumPrev1, PMac
c::StaticArray<picongpu::placeholder_definition14::position_pic::type, boost::mpl::inte
gral_c<uint32_t, 1U>>, boost::mpl::m_item<3L, picongpu::placeholder_definition17::weigh
ting, PMacc::StaticArray<picongpu::placeholder_definition17::weighting::type, boost::mp
l::integral_c<uint32_t, 1U>>, boost::mpl::m_item<2L, picongpu::placeholder_definition15
::momentum, PMacc::StaticArray<picongpu::placeholder_definition14::position_pic::type, 
boost::mpl::integral_c<uint32_t, 1U>>, boost::mpl::m_item<1L, picongpu::placeholder_def
inition12::position<picongpu::placeholder_definition14::position_pic, PMacc::placeholde
r_definition11::pmacc_isAlias>, PMacc::StaticArray<picongpu::placeholder_definition14::
position_pic::type, boost::mpl::integral_c<uint32_t, 1U>>, boost::mpl::map<boost::mpl::
na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boo
st::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl
::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, boost::mpl::na, b
oost::mpl::na, boost::mpl::na, boost::mpl::na>>>>>>, PODType=PMacc::math::AlignedData]"

$PICSRC/src/libPMacc/include/particles/memory/frames/Frame.hpp(73):
 here
            instantiation of class "PMacc::Frame<T_CreatePairOperator, T_ParticleDescri
ption> [with T_CreatePairOperator=PMacc::ParticlesBuffer<PMacc::ParticleDescription<boo
st::mpl::string<101, 0, 0, 0, 0, 0, 0, 0>, picongpu::SuperCellSize, picongpu::DefaultAt
tributesSeq, picongpu::ParticleFlagsElectrons, boost::mpl::vector2<picongpu::Communicat
ionId<5U>, picongpu::MemoryFactor<2UL>>>, picongpu::SuperCellSize, 3U>::OperatorCreateP
airStaticArray<1U>, T_ParticleDescription=PMacc::ParticleDescription<boost::mpl::string
<101, 0, 0, 0, 0, 0, 0, 0>, picongpu::SuperCellSize, boost::mpl::vector5<picongpu::plac
eholder_definition12::position<picongpu::placeholder_definition14::position_pic, PMacc:
:placeholder_definition11::pmacc_isAlias>, picongpu::placeholder_definition15::momentum
, picongpu::placeholder_definition17::weighting, picongpu::placeholder_definition16::mo
mentumPrev1, PMacc::placeholder_definition8::localCellIdx>, picongpu::ParticleFlagsElec
trons, boost::mpl::vector2<picongpu::CommunicationId<5U>, picongpu::MemoryFactor<2UL>>>
]" 
$PICSRC/src/libPMacc/include/particles/memory/buffers/ParticlesBuffer.hpp(121): here
...

[psychocoderHPC]

I found the problem and wrote a quick fix.
You can find the fix here

After the fix there are still warnings in:

• icc warning #3415 in Tuple.hpp (a few thousand times)
• icc warning #1624

boost/1.57.0-intel2013-sp1/include/boost/mpl/map/aux_/item.hpp(117): warning #1624: "char (&operator||(const boost::mpl::m_item_<Key, T, Base> &, boost::mpl::aux::type_wrapper<Key> *))[boost::mpl::next<Base::order>::type::value]" declares a non-template function -- add <> to refer to a template instance
  friend order_tag_ operator||(const m_item_ &, ::boost::mpl::aux::type_wrapper< Key>  *)

If I have more time I finish the pull request and push it to dev

[ax3l]

greet, feel free to already open the pull request and put it on WIP :sparkles:

you can probably not fix the warnings coming from boost headers, that is actually a cmake issue

[ax3l]

Currently we still have a problem with PMacc tuples and icc.

@erikzenker @Flamefire do you want to take a look pls?

[psychocoderHPC]

IMO we should not start to fix it before the last C++98/03 release is out.

reasons for that:

• maybe this problem is solved with C++11
• with C++11 we can remove the BOOST_PP which complicated a fix see my fix three comment above
• maybe we can save a lot of work and use boost fusion or native c++11 tuple (both were not available in the past)
• the intel compiler is currently not as important (not speed up the gpu part) but will be important with PIConGPU-alpaka

[psychocoderHPC]

I missed that @ax3l links from #1426 to this issue.

[ax3l]

yes makes sense to fix it with C++11 instead of BOOST_PP magic :+1:

[ax3l]

@psychocoderHPC we could check this again before the 0.3.0 release, but "official" support will need to wait unti we have our new CI up and running I guess.

[psychocoderHPC]

Yes you can create the release candidate and we need than check this and the clang compile again.

[psychocoderHPC]

Tested the release candidate 0.3.0-rc1 with the following modules on hypnos-hzdr.

• [x] runtime test KHI (no HDF5 and no ADIOS)

$ module li
Currently Loaded Modulefiles:
  1) mpfr/3.1.2                    4) intel/15.2                    7) openmpi/1.8.6.kepler.cuda80
  2) mpc/1.0.1                     5) cmake/3.3.0                   8) boost/1.60.0
  3) gmp/5.1.1                     6) cuda/8.0                      9) valgrind/3.8.1