Picongpu: Agreement between h5 and binary results

[cbontoiu]

Hello,

FIRST PROBLEM

I have run two identical and independent simulations writing out results with the adios plugin and respectively with the hdf5 plugin. A linearly polarized laser pulse hits a thick tube made of singly-ionized carbon atoms.

Now I am trying to make sense of the comparison for the electric and magnetic fields between the two in the axial yx-plane, so z = 0. For the hdf5 results, I use the slider of the openPMD viewer while for the adios results I use a Python syntax like for example, using the z-th cell that corresponds to z = 0.
values, factors = get_EX_field(ts, it); ts.flush(); A = values[zCell] * factors * 1e-12 # [TV/m] Z Y X
based on the definitions written in this file
adios_quantities.txt
with an example

Note that I need to multiply values with factors in order to obtain the correct values (presumably in SI) and as you can see below there is a perfect agreement for all electric field components.
ADIOS results are here

HDF5 results are here

The problem appears with the magnetic fields
ADIOS results are here

HDF5 results are here

It might happen that the Bx and By are just noise or remnants of the laser-matter interaction, that is, if there was no target, only Bz should exist due to laser polarization in the x-plane, but still to regard 450 T as noise is not easy to accept. Then, if this is the case, why there are differences between the output of the two plugins. The mesh was the same, everything was the same except for the plugin used.

SECOND PROBLEM

As I am using moving mesh with 2 GPUs, the laser pulse progresses beyond the y = 1575 nm, which one GPU holds. The openPMD viewer manages the get the absolute y-coordinates, maybe because they are embedded in the h5 results but in my case, I struggle to retrieve these coordinates x,y,z from the output. This is more difficult and I have no clue how to do it because the syntax must be different than for a vector field. Meanwhile, I am forced to use only relative y-coordinates from y = 0 to y = 1575 nm.

The same goes for the corresponding velocities ux, uy, uz and energy w.

Any idea how to achieve this?
Any help is greatly appreciated.

Kind regards and thank you for your time.
Cristian

[sbastrakov]

Hello @cbontoiu ,

Could you clarify which PIConGPU version you used and attached the .cfg files for your runs? FYI we do no thave native HDF5 plugin in the dev branch any more (and output to .h5 files is done via the openPMD-API plugin). Nevertheless, it makes sense to looks into that since in case there were some issues they may still be present elsewhere.

Regarding the second question, I think it is controlled by attribute gridGlobalOffset as described by the openPMD standard. So this should be a property (or however HDF5 and ADIOS calls it technically) of each dataset containing mesh data. This should be true regardless of which PIConGPU plugin you used for the output.

[cbontoiu]

Hello @sbastrakov and thank you for your quick reply.
I think I use version 0.5, not the dev version.

The _output log file at compilation_ is here

out_log.txt

the _output log file from runtime_ is here

output.txt

and the _cfg file_ is here

runConfiguration.txt

_Note:_ These three files attached correspond to another model in which there is not target so using InitPipeline = bmpl::vector< >; otherwise everything else is the same.

[PrometheusPi]

@cbontoiu
Regarding your fisrt question: Could you please specify where you see differences in the magnetic field between ADIOS and HDF5? Bz looks very similar and due to the noise in Bx and By, I would not wonder why there are differences by a factor 2. (you see a similar effect with Ez.) Such differences in noise are common in parallel PIC codes and are also seeded intrinsically if you used random start positions within a cell.

Regarding you second question: @sbastrakov input is the correct solution. Please use the suggested attribute. I understand why positions will be shifted, but why does this moving window effects your "velocities ux, uy, uz and energy w"? (By the way, w is the weighting.)

[cbontoiu]

Thank you @sbastrakov and @PrometheusPi for your kind replies. There is no problem with the velocities, I just didn't know how to extract them from the adios data bundle as I didn't know how to do it for the position coordinates, but now I will dig into the openPMD-standard.