Picongpu: Restart issue with background fields continues to exist

Created on 14 Jul 2017 · 12Comments · Source: ComputationalRadiationPhysics/picongpu

This is a follow up to #2113:

Test with the TWTS/plane wave background field setup (based on dev) still showed disappearing particles and wrong field values when used with restarts. Thus a restart issue still exists.

UPDATE/important to mention (thanks @ax3l for pointing this out)
This is (most likely) an issue with the field restart! The disappearing particles is the most obvious result, but is probably only caused by the wrong field values after restart.

UPDATE No. 2:
I forgot to cherry-pick #2112 aee7784672da6c3ded6fe00f47be9fdcd4b97948 - I know updated the plots and the error is about a factor 2 in the simple setup presented below.

The following difference could be seen for the same simulation setup - fist with restart every 10kth iteration, the second without any restart.

n_macroparticles

fieldenergy

As mentioned by @psychocoderHPC in #2113 I will try to find an easy setup to reproduce this bug based on 0.3.0.

mentioning:
@BeyondEspresso
@steindev

bug plugin

Source

PrometheusPi

All 12 comments

@PrometheusPi checkpoint/restart with ADIOS or HDF5?

ax3l on 14 Jul 2017

@ax3l Restart with HDF5.

PrometheusPi on 14 Jul 2017

👍1

@psychocoderHPC I could reproduce this error using 0.3.0 with a cherry picked daf0d002704b66abf6192995e95f5f7cfc73a30e (#2113):

UPDATE No. 2:

update pictures - error now much larger
cherry-picked the plugin now, too #2112

Based on the default Laser Wakefield example (8 GPUs), but without particles and without Laser I used the following background field:

namespace picongpu
{
    class FieldBackgroundE
    {
    public:
        /* Add this additional field for pushing particles */
        static constexpr bool InfluenceParticlePusher = true;

        /* We use this to calculate your SI input back to our unit system */
        PMACC_ALIGN(m_unitField, const float3_64);

        HDINLINE FieldBackgroundE( const float3_64 unitField ) : m_unitField(unitField)
        {}

        /** Specify your background field E(r,t) here
         *
         * \param cellIdx The total cell id counted from the start at t = 0
         * \param currentStep The current time step */
        HDINLINE float3_X
        operator()( const DataSpace<simDim>& cellIdx,
                    const uint32_t currentStep ) const
        {
            /* example: periodicity of 20 microns ( = 2.0e-5 m) */
            constexpr float_64 period_SI(20.0e-6);
            constexpr float_64 omega_SI = 2.0 * PI * 2.9979e8 / period_SI;
            /* calculate cells -> SI -> m to microns*/
            const float_64 y_SI = cellIdx.y() * SI::CELL_HEIGHT_SI;
            /* note: you can also transform the time step to seconds by
             *       multiplying with DELTA_T_SI */

            /* specify your E-Field in V/m and convert to PIConGPU units */
            const float_X sinArg = precisionCast<float_X>( y_SI / period_SI * 2.0 * PI - currentStep *SI::DELTA_T_SI * omega_SI);
            return float3_X(0.0, math::sin( sinArg ) / m_unitField[1], 0.0);
        }
    };

and equivalent for the magnetic field.

There is an obvious ~~(but in this case small)~~ difference in the field energy:
fieldenergy_0 3 0

Due to the periodic nature of these spikes, I will re-run a simulation without moving window and see if the spikes disappear.

PrometheusPi on 14 Jul 2017

👍1

This bug persist even without moving window:

Update No. 2:
Update after cherry-pick -error now much larger

fieldenergy_0 3 0_nomoving

PrometheusPi on 14 Jul 2017

The error does not occur for normal/standard laser input (in this case plane wave - with periodic (transversal) boundary conditions and moving window) which is treated by the Yee solver.

fieldenergy_0 3 0_planewave