Picongpu: mpiexec stderr: unrecognised option '--e_png.period'

[BifengLei]

I am running a simulation (copied from LWFA example) with 0.4.1 on Hypnos, with the k80.tpl provided by the code. openmpi/1.8.6.kepler.cuda80 is used. Unfortunately, the job is always killed by such err:
[1,10]<stderr>:unrecognised option '--e_png.period'

Can you please figure out what is maybe wrong here?

---

Solution: you have not provided the PNGwriter dependency or the simulation has no species "e" (speciesDefinition.param). Here are the official Hypnos (HZDR) profiles to copy from.

[sbastrakov]

Did you use one of the standard .cfg files (for -c option in tbg) that come with the example? If so, could you please clarify which one, or otherwise attach your file?

[berceanu]

Just to clarify -- @BifengLei, your job submission command probably looks like this:
tbg -s qsub -c etc/picongpu/<XX>.cfg -t etc/picongpu/hypnos-hzdr/k80.tpl $SCRATCH/runs/<output folder>

Could you please attach the file etc/picongpu/<XX>.cfg that you are using? Here XX can be 4, 8, 16 etc.

[BifengLei]

My .cfg file is attached (please delete suffix .txtfor use). It is directly modified from standard 8.cfg file for the 2D case.
8.cfg.txt

[berceanu]

I thought the dimensionality of the the simulation is set in include/picongpu/param/dimension.param, which for 2D should be

#define PARAM_DIMENSION DIM2

and then TBG_devices_z is automatically ignored.

[BifengLei]

Yes, in dimension.param the dimensionality has been set to DIM2.

So, you mean, the TBG_devices_zshould still be set in.cfg file?

[berceanu]

Could you please also attach the stderr and stdout files from $SCRATCH/runs/<output folder>?

[berceanu]

So, you mean, the TBG_devices_zshould still be set in.cfg file?

Yes, TBG_devices_z, as well as the last component of TBG_gridSize should be automatically ignored when setting PARAM_DIMENSION DIM2 in dimension.param.

[BifengLei]

The output files are attached.
stdout.txt
stderr.txt

[sbastrakov]

@BifengLei unfortunately, so far I did not manage to reproduce your issue. I've taken 0.4.1 version, the standard LWFA example there, only changed dimension to DIM2 in dimension.param and ran with your attached .cfg file on k80 hypnos queue. For me the simulation is actually starting and not crashing. Am I still somehow deviating from your setup?

Sorry to ask for additional info again, but could you please also provide an archive of the tbg subdirectory of your output directory? It is on the same level as e.g. simOutput directory and stdout/stderr files, and contains files like submit.cfg, submit.start, etc.

I believe TBG_devices_z stuff is not related to the issue and you could actually omit that in 2D as long as you don't use this variable yourself (and you don't, so that should be fine).

[BifengLei]

@sbastrakov thank you so much for your help.
All the output files are attached.
To mention that I also define my new density profile as attached. Can you please also have a look?
simOutput.zip

[sbastrakov]

Thanks to @psychocoderHPC , it seems you compiled picongpu without png, and so it does not recognize the png plugin options. Unfortunately, we could not check it immediately from the data provided, but seems likely. To check please run module list after you've loaded the picongpu profile and attach the output. And please also provide output of ./bin/picongpu --help ran from your input directory (where picongpu was built).

[BifengLei]

Currently Loaded Modulefiles:

  1) gcc/5.3.0              4) psm/3.1                7) zlib/1.2.8            10) hdf5-parallel/1.8.15
  2) cmake/3.10.1           5) openmpi/1.8.6.kepler.cuda80         8) numactl/2.0.7         11) libsplash/1.6.0
  3) cuda/8.0               6) boost/1.65.`           9) pngwriter/0.7.0

The./bin/picongpu --help gives
./bin/picongpu: error while loading shared libraries: libboost_filesystem.so.1.62.0: cannot open shared object file: No such file or directory

I am not sure why this happens on hypnos. I will check it out.

[sbastrakov]

@BifengLei thanks . The module list actually looks fine to me.

There is another way to run ./bin/picongpu --help to hopefully avoid the issue you are getting. You will need to allocate a node in interactive mode and run the command from there - this way the shared libraries should be resolved properly. To get a node please run
qsub -I -q k80j -lwalltime=00:30:00 -lnodes=1:ppn=16
and then wait a little bit till you are redirected to the allocated node. (As part of your k80 pincongpu profile there is getNode command that does exactly the same but for the k80 queue.)
After you are on the new node (then your terminal has smth like [user_name]@kepler[number] instead of [user_name]@hypnos4) source your picongpu profile again, go to your input directory and try running ./bin/picongpu --help .

[BifengLei]

@sbastrakov By the way, in the new version, the minimum libsplash is required to be 1.7.0+., is it OK if I just use 1.6.0 since there seems only libsplash/1.6.0 available for gnu530 chain.

[sbastrakov]

Ah, good catch, I did not notice that. Can't tell right now if the 1.6.0 is still supported, but I guess there has been a reason to increase the min required version (will clarify).

Is there any reason the standard profile does not work for you? For hypnos k80 it has gcc 4.9.2 and libsplash 1.7.0, which for me builds fine out-of-the box.

[BifengLei]

The reason is that, once I try to use gcc4.9.2, I have to use openmpi/1.8.6 but not openmpi/1.8.6.kepler.cuda80. Then, there is always a GPU problem as attached.

stderr.txt

I am wondering if I could have a look for your picongpu.profile.

[sbastrakov]

@BifengLei sorry for ambiguous writing. By the standard profile I meant the one in the picongpu repository, file etc/picongpu/hypnos-hzdr/k80_picongpu.profile.example. The difference I see from your setup is that the profile uses openmpi/2.1.2.cuda80 module, not openmpi/1.8.6.

[BifengLei]

@sbastrakov I am so sorry that I actually did not recognize the updated picongpu.profile here. Now, everything works perfectly.