Ompi: UCX request not returned by MPI_Cancel

[devreal]

I'm posting a bunch of requests (both through MPI_Isend and MPI_Irecv), which are tested periodically and reposted. Eventually, all requests are destroyed using MPI_Cancel. When using Open MPI 4.0.x and UCX (1.6.x) I get the following warning:

$ mpirun -n 1 ./test_mpi_cancel
[1566305465.736988] [tauruslogin3:9066 :0]          mpool.c:38   UCX  WARN  object 0xc01e80 was not returned to mpool ucp_requests

It seems that MPI_Cancel does not return the request object to UCX.

Here is a simple reproducer:

#include <mpi.h>

int main(int argc, char **argv)
{
  MPI_Init(&argc, &argv);
  int val;
  MPI_Request req;
  MPI_Irecv(&val, 1, MPI_INT, 0, 1001, MPI_COMM_WORLD, &req);
  MPI_Cancel(&req);
  MPI_Finalize();

  return 0;
}

Open MPI version:

$ ompi_info | grep revision
  Open MPI repo revision: v4.0.0rc4-452-gf96994b

UCX version:

$ ~/opt/ucx-1.6.x/bin/ucx_info -v
# UCT version=1.6.1 revision 736d503

[brminich]

According to MPI standard, you'd still need to call MPI_Request_free. (mpi 3.1, chapter 3.8.4 Cancel)

It is still necessary to call MPI_REQUEST_FREE,
MPI_WAIT or MPI_TEST (or any of the derived operations) with the cancelled request as
argument after the call to MPI_CANCEL. If a communication is marked for cancellation,
then a MPI_WAIT call for that communication is guaranteed to return, irrespective of
the activities of other processes (i.e., MPI_WAIT behaves as a local function); similarly if
MPI_TEST is repeatedly called in a busy wait loop for a cancelled communication, then
MPI_TEST will eventually be successful.

[devreal]

Ahh right, my bad. Sorry for the noise!