Nservicebus: CancellationToken support for EndpointInstance, MessageSession and/or handler context

[danielmarbach]

Relates to https://github.com/Particular/NServiceBus/issues/3765

As it stands right now the high-level APIs like endpoint instance, message session and handler context do not provide cancellation support via CancellationToken. We have been discussing it in the past but implicitly decided not to do it for v6 and add cancellation support as a feature for minor versions. While this is an acceptable approach, I would like to highlight a few ideas around cancellation support which would enable us to introduce the APIs and gradually extend the cancellation support as we move on.

We currently have three levels which could benefit from cancellation:

• EndpointInstance Start and Stop
• MessageSession members
• Handler Context members

Before you read on let's agree on the definition of cancellation:

Starting with the .NET Framework 4, the .NET Framework uses a unified model for cooperative cancellation of asynchronous or long-running synchronous operations. [...]
The new cancellation model makes it easier to create cancellation-aware applications and libraries, and it supports the following features:

• Cancellation is cooperative and is not forced on the listener. The listener determines how to gracefully terminate in response to a cancellation request.
• Requesting is distinct from listening. An object that invokes a cancelable operation can control when (if ever) cancellation is requested.
• The requesting object issues the cancellation request to all copies of the token by using just one method call.
• A listener can listen to multiple tokens simultaneously by joining them into one linked token.
• User code can notice and respond to cancellation requests from library code, and library code can notice and respond to cancellation requests from user code.
• Listeners can be notified of cancellation requests by polling, callback registration, or waiting on wait handles.

https://msdn.microsoft.com/en-us/library/dd997364(v=vs.110).aspx

Endpoint Instance

We had various discussions in the past whether it makes sense or not to pass in CancellationToken to the Start and Stop method of the endpoint. The question was also raised by @SeanFeldman in slack.

For example in Service Fabric, the hosting environment passes in a CancellationToken which will be canceled after a period before the process gets killed. An example of such an interface can be seen here

https://msdn.microsoft.com/en-us/library/microsoft.servicefabric.services.communication.runtime.icommunicationlistener.aspx

The most important aspect that we have to take into consideration is that we won't lose messages in flight. Passing in a cancellation token to start and stop would allow the hosting environment to put up some "SLA" by saying: I pass you the token which you can observe. I'll cancel it if I decide that it took too long to start the endpoint.

I know this is controversial but I'll show it here for completeness sake.

Message Session

Until we introduce UoW scoped session all message session operations are immediate dispatch and call into potentially long running operations like connections, raw native transport operations and more. A cancellation token here would be very convenient.

Handler Context

This one is a bit special. Handler Context operations are by default batched. But you can opt-out from it by setting the immediate dispatch flag on the send options. Nonetheless even for batched dispatched operations, we are executing parts of the outgoing pipeline which can be extended/customized with user code what can call into long operations as well. Applying the same principles like for endpoint instance would simplify the API. I think we should not bury the cancellation token behind extensions methods like RequireImmediateDispatch.

Questions

• Should we provide the overloads with cancellation token support already in v6 but gradually extend the cooperative cancellation support in minor releases? Remember we now have interfaces for message session, endpoint instance, etc. Changing those interfaces is a breaking change.
• How do we want to expose the cancellation tokens? The simplest and most pragmatic way would be to accept a token for each message operations. The users can decide if they want to use multiple tokens, linked tokens or just one token for all operations (or even the one passed in by the pipeline)?
• Should we treat cancellation for batched dispatch differently from immediate dispatch?
  

  Code snippets

@andreasohlund and I spiked a few code snippets as ideas, discussion starters

Handler(msg,context)
{
  MyWebApiFacad.Call(new Request(), context.CancellationToken);

  if(...)
     context.Abort();
}

Assumption: Batched dispatch operations always should be canceled together:

Handler(msg,context)
{

   context.CancellationTokenForBatchedDispatch(myToken);

   context.Send(new X());
}

Immediate dispatch controls the scope of cancellation

Handler(msg,context)
{
   var options = new SendOptions();

   option.ImmediateDispatch(myToken);

   context.Send(new X(),options);
}

Pragmatic way (goes with .NET standards):

Handler(msg,context)
{
   var token = Context.CancellationToken;

   await context.Send(new ProvisionVideo(), token);
   await context.Publish(new ProvisonStartedEvent(), token);
}

Config driven cancellation of inflight during shutdown:

endpointConfig.CancelMessagesInFlightAtShutdownAfter(Timespan.Seconds(2))

Session or scope level cancellation:

WebApiGet()
{
    await endpointInstance.Send(new Z(),token);

    context.Use(session=>{
       await session.Send(new X());
       await session.Send(new Y());
   },token);
}

[danielmarbach]

@Particular/nservicebus-maintainers Thoughts? Let's align and make the decision whether we need this for v6 or not.

[timbussmann]

I don't see as cancellation support bringing such a huge value to the table that it needs to be included in the 6.0 release. Not a big fan of adding the parameters without actually using them.

[danielmarbach]

It could mean we'll have to wait for next major then. And like I said, fan
or not, the concept cooperative cancellation would allow us to build the
functionality step by step

[SeanFeldman]

If releasing another major will happen fast, perhaps not an issue. But history tells us it's not the case, so what @danielmarbach is suggesting sounds right.

[bording]

What are the scenarios around cancellation that make sense for us? Is this just about allowing a way to use a cancellation token to shut down an endpoint? Is trying to cancel an individual message send even something we'd want to try and support?

I'm all for supporting cancellation where it makes sense, but I don't yet see the benefit in trying to say we support cancellation everywhere, even for individual message sends.

[timbussmann]

It could mean we'll have to wait for next major then.

I'm good with that.

What are the scenarios around cancellation that make sense for us?

I was thinking about this too. From our perspective it's mostly shutting down the bus I think. From a user perspective there may be send scenarios where you want to cancel after a certain time (e.g. when the broker is unavailable and the default timeout is not good enough?) [completely made that up, not sure whether that really is a scenario]. How many of the used transport/persistence async APIs actually accept a cancellation token?

[distantcam]

Could we add the cancel versions as overloads? That way we wouldn't need to wait for the next major.

I'd rather wait and do it right than rush this now.

[danielmarbach]

@distantcam I specifically would like to just add the overloads in this version but not implement it and then gradually implement the mechanics underneath in coming minors.

[distantcam]

Not implement it? I don't like that idea.

[danielmarbach]

Let me rephrase:

Implement as much as we think it makes sense for this release.

Most pragmatic simple one would be to not invoke the underlying operation if the token is cancelled (early exit). Depending on how much we want to invest we can implement more or leave out the work for future minors.

[danielmarbach]

Currently tagged as future. We will revisit this when the dust around the new async API is settled

[dbelcham]

Closing this since the discussion is over and there is no action to be taken. Re-open when the time to address/consider this arrives.

[SimonCropp]

Currently tagged as future. We will revisit this when the dust around the new async API is settled

@andreasohlund @danielmarbach perhaps time to revisit?

[andreasohlund]

@SimonCropp any specific use case on your end that would require this?

[SimonCropp]

when a endpoint is stopped would be nice to be able to cancel pending tasks and avoid a service timeout

[mikesigs]

I am having an issue that seems related, and having Cancellation support might help.

I have a Request/Response pattern implemented, where the request handler is making an HTTP request with HttpClient. The HttpClient timeout defaults to 100 seconds. However, the default timeout for the TransactionScope is 60 seconds. So if the HTTP request takes longer than 60 seconds, the transport fails when sending the response because the transaction has timed out.

If there was a cancellation token passed to the Handle method, I could pass that to the HttpClient when making the request. I could also check that token in my user code to see if I need to handle the cancellation.

[bording]

@mikesigs For that scenario, I'd recommend extending the transaction timeout to cover the duration of your http requests.

[SimonCropp]

@bording that would still likely result in a service shutdown/restart failure if a message is being processed at that time.

[SimonCropp]

bump

[timbussmann]

@SimonCropp is the bump related to a specific question or the issue in general? As for the moment, we don't see cancellation token being a high priority feature to be added to our APIs. Technically it would indeed make sense, but the benefit vs. effort doesn't seem to make this a high priority issue at the moment. Have you considered using your own CancellationTokenSource and cancelling that after calling Stop() on the endpoint?

As for @mikesigs scenario, that seems to be a quite different scenario and more complicated in the regards that we're also not actively tracking transaction timeouts anyway. I think @bording has described a valid workaround and there might be other approaches as well (e.g. creating a custom token with a timeout when invoking the handler and passing that to the user code)

[SimonCropp]

@timbussmann the bump was because 27 days i asked a question that AFAIK has not been answered.

Even if people manage their own CancellationTokenSource that will not help with cancellable tasks that are managed by NSB, eg comms with the persistence or the transport.

[timbussmann]

if you talk about service shutdown failure because endpoints take too long to shutdown due to pending messages, yes that's still possible and not addressed by Brandon's suggestion.

[SimonCropp]

@timbussmann yep.

any chance of getting this scheduled?

[timbussmann]

@SimonCropp I'll have to forward this feature request to platdev, due to the amount of changes and involved public APIs.

[SimonCropp]

Any update?

[timbussmann]

Not so far. In the meanwhile, I understand that the issue for you seems to be service restart/shutdown failure due to processing inflight messages when shutting down, correct?

[SimonCropp]

yep

[timbussmann]

we're reopening this issue so it represents the state of this feature request. The prioritization will be handled internally but this issue will be updated once it has been added to the roadmap.

[SimonCropp]

any update?

[timbussmann]

@SimonCropp the issue is still in the backlog.