Openpmd-api: ADIOS1 deferred file creation

Created on 7 Jun 2019  Â·  3Comments  Â·  Source: openPMD/openPMD-api

Describe the bug
The ADIOS1 backend defers the creation of new files to the point where actual data is written.
It is possible to create workflows where between file creation and writing the first dataset, the backend will receive a OPEN_FILE IO task. The ADIOS1 backend will open the file in read mode. This will only succeed if the file exists not only logically, but also physically.

To Reproduce
Use the sample C++ writing example, but replace

    Series series = Series(
        "../samples/3_write_serial.h5",
        AccessType::CREATE
    );
    cout << "Created an empty " << series.iterationEncoding() << " Series\n";

    MeshRecordComponent rho =
      series
          .iterations[1]
          .meshes["rho"][MeshRecordComponent::SCALAR];

… with …

    Series series = Series(
        "3_write_serial%T.bp", // file-based layout, ADIOS1 backend
        AccessType::CREATE
    );
    cout << "Created an empty " << series.iterationEncoding() << " Series\n";
    series.iterations[1].setTime(0.5); // create file for iteration logically, but not physically
    series.iterations[0].setTime(0.5); // switch to other file to trigger an OPEN_FILE for switching back again

    MeshRecordComponent rho =
      series
          .iterations[1]
          .meshes["rho"][MeshRecordComponent::SCALAR];

Expected behavior
OPEN_FILE should succeed also with files that have been created logically only, but not physically.

ADIOS1 bug

All 3 comments

With #524 solved for HDF5, is this ADIOS1 issue still open/reproducible?

I tested it out, the above example finishes cleanly and the workaround in PIConGPU's openPMD plugin is apparently not necessary any longer either. Looks good.

Nice, thanks to @guj for helping tremendously with the fixes!

Was this page helpful?
0 / 5 - 0 ratings

Related issues

ax3l picture ax3l  Â·  7Comments

lucafedeli88 picture lucafedeli88  Â·  3Comments

KseniaBastrakova picture KseniaBastrakova  Â·  6Comments

KseniaBastrakova picture KseniaBastrakova  Â·  4Comments

C0nsultant picture C0nsultant  Â·  6Comments