I compiled kratos with
-DMPI_NEEDED=ON
Then I ran a Python Unit Test of the DEM Application. At the end of the test, it showed a Segmentation Fault.
Then I ran a regular case of the DEM Application and everything was fine.
I deleted the file libs/mpipython.so. Then all tests ran without problems.
maybe related to #77?
Maybe they are related. But I am not using mpirun. Just normal Python.
Seems that the error comes from calling mpi init without using mpirun. @maceligueta would check where the mpi init is run and tries to avoid it.
I partly found the reason why it is crashing. I am importing kratos_utilities.pyto delete a file, and this script has a try/except for importing Kratos.mpi. @roigcarlo and I are trying to change the try/except for another solution.
@maceligueta I guess this is my fault, I made this function. Didn't know it could lead to such an error...
Please let me know if I can help
should be fixed in #2375
@maceligueta sorry for the inconveniences!
@pooyan-dadvand @roigcarlo could this be somehow prevented with MPI_Initialized to not have this problem again in the future?
@jcotela and I had this problem before without finding a robust way to avoid it. Our long term plan was to create the MPI core.
@philbucher I'm afraid MPI_Initialized is a bit of a chicken-and-egg problem here... To check if MPI_Initialized requires importing something that links to mpi, and by the time you are importing it, it might be too late to check.
According to https://stackoverflow.com/a/12678625 there's no nice way to detect if a code was run with mpirun. However, checking the presence of certain environment variables does the work. One has to know at least one variable from each mpi library (OpenMPI, Intel,, ...) and expect them not to change them much. If we write a general function maintained by everyone it might work. At the DEM Application we are capable of detecting Intel and OpenMPI with normality.
I think that we should distinguished between the main script where we know if we are calling the MPI version which loads the MPI kernel and the rest of the code that we want to be somehow agnostic to MPI to avoid duplicated code.
This means that if we assume that we call a separate main script with mpirun then there we can safely set that we are running MPI and the rest of the scripts and especially the factories would check the Kerenl::IsMPI to select a correct object/behavior
I understand that this solution is better, @pooyan-dadvand , once the MPI kernel is ready.
Most helpful comment
I partly found the reason why it is crashing. I am importing
kratos_utilities.pyto delete a file, and this script has atry/exceptfor importing Kratos.mpi. @roigcarlo and I are trying to change thetry/exceptfor another solution.