Ompi: v4.0.1 breaks MPI 1.x compatibility

[ashwinvis]

Background information

What version of Open MPI are you using? (e.g., v1.10.3, v2.1.0, git branch name and hash, etc.)

v4.0.1

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Packaged by ArchLinux

Please describe the system on which you are running

• Operating system/version: Archlinux
• Computer hardware: Intel(R) Core(TM) i7-5500U
• Network type: Irrelevant

---

Details of the problem

Prevents mpi4py from building and potentially can affect other libraries and applications. See https://bitbucket.org/mpi4py/mpi4py/issues/121/pip-install-error for a related issue and an unreleased hotfix.

shell$ pip install mpi4py --no-cache

...
    building 'mpi4py.MPI' extension
    /bin/mpicc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fstack-protector-strong -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fstack-protector-strong -fno-plt -march=x86-64 -mtune=generic -O3 -pipe -fstack-protector-strong -fno-plt -fPIC -DHAVE_DLFCN_H=1 -DHAVE_DLOPEN=1 -I/scratch/avmo/opt/dbdccffb8240d80/include/python3.7m -c src/MPI.c -o build/temp.linux-x86_64-3.7/src/MPI.o
    In file included from src/mpi4py.MPI.c:591,
                     from src/MPI.c:4:
    src/mpi4py.MPI.c: In function ‘__pyx_f_6mpi4py_3MPI_del_Datatype’:
    src/mpi4py.MPI.c:15094:36: error: expected expression before ‘_Static_assert’
       __pyx_t_1 = (((__pyx_v_ob[0]) == MPI_UB) != 0);
                                        ^~~~~~
    src/mpi4py.MPI.c:15107:36: error: expected expression before ‘_Static_assert’
       __pyx_t_1 = (((__pyx_v_ob[0]) == MPI_LB) != 0);
                                        ^~~~~~
    src/mpi4py.MPI.c: In function ‘__pyx_pf_6mpi4py_3MPI_8Datatype_44Free’:
    src/mpi4py.MPI.c:68978:17: error: expected expression before ‘_Static_assert’
         __pyx_v_p = MPI_UB;
                     ^~~~~~
    src/mpi4py.MPI.c:68992:17: error: expected expression before ‘_Static_assert’
         __pyx_v_p = MPI_LB;
                     ^~~~~~
    src/mpi4py.MPI.c: In function ‘__pyx_f_6mpi4py_3MPI_Datatype2String’:
    src/mpi4py.MPI.c:76319:37: error: expected expression before ‘_Static_assert’
       __pyx_t_1 = ((__pyx_v_datatype == MPI_LB) != 0);
                                         ^~~~~~
    src/mpi4py.MPI.c:76332:37: error: expected expression before ‘_Static_assert’
       __pyx_t_1 = ((__pyx_v_datatype == MPI_UB) != 0);
                                         ^~~~~~
    src/mpi4py.MPI.c: In function ‘__pyx_pymod_exec_MPI’:
    src/mpi4py.MPI.c:165206:62: error: expected expression before ‘_Static_assert’
       __pyx_t_3 = ((PyObject *)__pyx_f_6mpi4py_3MPI_new_Datatype(MPI_UB)); if (unlikely(!__pyx_t_3)) __PYX_ERR(21, 862, __pyx_L1_error)
                                                                  ^~~~~~
    src/mpi4py.MPI.c:165220:62: error: expected expression before ‘_Static_assert’
       __pyx_t_3 = ((PyObject *)__pyx_f_6mpi4py_3MPI_new_Datatype(MPI_LB)); if (unlikely(!__pyx_t_3)) __PYX_ERR(21, 863, __pyx_L1_error)
                                                                  ^~~~~~
    error: command '/bin/mpicc' failed with exit status 1

[jsquyres]

That is both correct and expected.

Please:

• Help us by strongly encouraging legacy application maintainers to stop using these constructs that were deprecated in 1996.
• See https://www.open-mpi.org/faq/?category=mpi-removed for more information.

If you need to, you can ./configure Open MPI v4.0.x with --enable-mpi1-compatibility, which will restore the prototypes for these symbols back into mpi.h.

NOTE: This --enable-mpi1-compatibility switch will disappear in a future release series of Open MPI.

[ashwinvis]

I understand that the deprecation was intended. However this breaking change came unexpected. I agree with the mpi4py developer, that this is not conforming to usual semantic versioning practices as the deprecation was incorporated in a patch / bug-fix release.

As a result this went out the radar of other maintainers. Maybe I am wrong but I don't see it mentioned in the release notes (https://www.mail-archive.com/[email protected]/msg00122.html). So it is understandable that package maintainers went ahead and build openmpi 4.0.1 without ensuring compatiblity.

See build history of mpi4py and openmpi:

• https://git.archlinux.org/svntogit/community.git/log/trunk?h=packages/python-mpi4py
• https://git.archlinux.org/svntogit/packages.git/log/trunk?h=packages/openmpi

I can only request the open-mpi team to follow versioning strictly (no breaking in patch releases) and communicate breaking changes in release notes in the future.

[jsquyres]

Agreed: we messed up in v4.0.0. MPI_UB (and friends) was not supposed to be in mpi.h in v4.0.0.

That was specifically announced in the 4.0.0 release notes, in the README, on our web site, in public talks that we gave before the v4.0.0 release, was called out to packagers before the v4.0.0 release, etc. We tried very hard to announce this ahead of time because we were very aware of the disruption that it would cause.

Unfortunately, we goofed in v4.0.0 and MPI-1-legacy-construct-using-applications continued to compile. So our warnings largely went unheeded, I guess...?

In v4.0.1, we fixed the bug. If you look at it one way, yes, we violated semantic versioning. If you look at it another way, we fixed the bug -- i.e., we did what we said we were going to do in v4.0.0.

[ashwinvis]

Thank you for the clarification and being honest about it. I agree that sufficient time was provided to library and application developers. Mistakes can happen at all ends.

Thanks again for the awesome library which fuels a lot of research around the world.

[jsquyres]

Thanks for your patience and understanding!